Sep 14 12:56:21.442267 ip-10-0-135-35 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 14 12:56:21.442282 ip-10-0-135-35 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 14 12:56:21.442292 ip-10-0-135-35 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 14 12:56:21.442509 ip-10-0-135-35 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 14 12:56:31.690886 ip-10-0-135-35 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 14 12:56:31.690903 ip-10-0-135-35 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 49288f158d244fe2b50864a8fd596767 --
Sep 14 12:58:24.268906 ip-10-0-135-35 systemd[1]: Starting Kubernetes Kubelet...
Sep 14 12:58:24.685561 ip-10-0-135-35 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 12:58:24.685561 ip-10-0-135-35 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 14 12:58:24.685561 ip-10-0-135-35 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 12:58:24.685561 ip-10-0-135-35 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 14 12:58:24.685561 ip-10-0-135-35 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 12:58:24.687588 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.687484    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 14 12:58:24.692082 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692064    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 12:58:24.692082 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692083    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692087    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692091    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692094    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692097    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692100    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692103    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692107    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692111    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692115    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692118    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692129    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692132    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692134    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692137    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692140    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692143    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692145    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692148    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 12:58:24.692158 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692151    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692153    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692156    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692159    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692162    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692165    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692168    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692171    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692174    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692176    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692179    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692181    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692184    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692186    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692189    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692191    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692194    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692196    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692199    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 12:58:24.692699 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692201    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692204    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692206    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692210    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692213    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692215    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692217    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692220    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692222    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692225    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692227    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692231    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692235    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692237    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692240    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692244    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692247    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692250    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692253    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692255    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 12:58:24.693166 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692258    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692261    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692264    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692266    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692269    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692273    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692275    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692278    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692281    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692284    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692286    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692289    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692292    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692295    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692297    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692300    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692304    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692307    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692309    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692312    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 12:58:24.693674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692315    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692317    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692321    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692324    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692326    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692329    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692332    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692335    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692338    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692341    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692344    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692346    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692349    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692353    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692357    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692360    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692363    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692366    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692369    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.692372    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 12:58:24.694199 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693828    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693837    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693841    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693846    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693850    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693853    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693856    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693858    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693861    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693865    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693867    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693870    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693873    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693875    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693878    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693881    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693884    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693886    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693889    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693893    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 12:58:24.694703 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693895    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693899    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693901    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693904    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693907    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693909    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693912    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693915    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693918    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693921    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693924    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693926    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693929    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693932    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693935    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693937    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693941    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693944    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693946    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693949    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 12:58:24.695220 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693952    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693954    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693957    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693960    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693962    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693966    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693970    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693972    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693975    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693978    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693980    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693983    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693986    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693989    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693991    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693994    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693997    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.693999    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694002    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 12:58:24.695718 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694004    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694007    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694012    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694015    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694018    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694021    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694025    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694028    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694031    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694034    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694037    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694040    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694043    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694046    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694049    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694052    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694055    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694058    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694060    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 12:58:24.696196 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694063    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694065    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694068    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694070    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694073    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694076    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694079    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694081    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694084    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694086    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694089    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694092    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694094    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694097    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694099    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694102    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694104    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694107    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694109    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694112    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 12:58:24.696672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694114    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694526    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694532    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694537    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694540    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694544    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694546    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694549    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694552    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694556    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694559    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694561    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694564    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694566    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694568    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694571    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694589    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694592    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694594    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 12:58:24.697162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694597    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694600    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694602    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694605    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694608    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694610    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694613    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694616    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694618    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694621    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694623    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694626    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694628    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694631    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694634    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694637    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694639    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694642    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694644    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694647    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 12:58:24.697668 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694649    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694652    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694655    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694666    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694669    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694672    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694674    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694677    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694680    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694683    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694685    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694688    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694690    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694693    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694695    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694698    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694700    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694703    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694706    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694709    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 12:58:24.698170 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694711    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694714    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694717    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694719    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694721    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694724    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694727    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694730    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694732    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694736    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694740    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694743    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694745    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694748    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694750    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694753    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694764    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694766    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694769    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 12:58:24.698672 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694773    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694776    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694779    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694782    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694785    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694788    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694791    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694793    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694796    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694799    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694801    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694804    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694806    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694809    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694811    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694814    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694816    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694819    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694821    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694824    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 12:58:24.699184 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694826    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.694828    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.694907    2581 flags.go:64] FLAG: --address="0.0.0.0"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.694939    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.694946    2581 flags.go:64] FLAG: --anonymous-auth="true"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.694950    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.694955    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.694959    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.694963    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.694968    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.694985    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.694988    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.694992    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.694995    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.694998    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695001    2581 flags.go:64] FLAG: --cgroup-root=""
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695004    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695007    2581 flags.go:64] FLAG: --client-ca-file=""
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695010    2581 flags.go:64] FLAG: --cloud-provider="external"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695013    2581 flags.go:64] FLAG: --cluster-dns="[]"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695018    2581 flags.go:64] FLAG: --cluster-domain=""
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695021    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695024    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695027    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 14 12:58:24.699701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695030    2581 flags.go:64] FLAG: --container-log-max-files="5"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695038    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695041    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695044    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695047    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695050    2581 flags.go:64] FLAG: --contention-profiling="false"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695053    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695056    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695060    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695063    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695071    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695074    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695077    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695080    2581 flags.go:64] FLAG: --enable-load-reader="false"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695083    2581 flags.go:64] FLAG: --enable-server="true"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695087    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695091    2581 flags.go:64] FLAG: --event-burst="100"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695094    2581 flags.go:64] FLAG: --event-qps="50"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695097    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695100    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695104    2581 flags.go:64] FLAG: --eviction-hard=""
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695107    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695110    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695113    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695116    2581 flags.go:64] FLAG: --eviction-soft=""
Sep 14 12:58:24.700264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695119    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695122    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695125    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695128    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695131    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695134    2581 flags.go:64] FLAG: --fail-swap-on="true"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695137    2581 flags.go:64] FLAG: --feature-gates=""
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695141    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695144    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695147    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695150    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695153    2581 flags.go:64] FLAG: --healthz-port="10248"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695156    2581 flags.go:64] FLAG: --help="false"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695159    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-135-35.ec2.internal"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695162    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695165    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695168    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695173    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695176    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695179    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695182    2581 flags.go:64] FLAG: --image-service-endpoint=""
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695185    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695189    2581 flags.go:64] FLAG: --kube-api-burst="100"
Sep 14 12:58:24.700885 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695192    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695195    2581 flags.go:64] FLAG: --kube-api-qps="50"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695198    2581 flags.go:64] FLAG: --kube-reserved=""
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695201    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695204    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695208    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695211    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695213    2581 flags.go:64] FLAG: --lock-file=""
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695217    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695219    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695222    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695228    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695231    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695234    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695237    2581 flags.go:64] FLAG: --logging-format="text"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695240    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695243    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695246    2581 flags.go:64] FLAG: --manifest-url=""
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695249    2581 flags.go:64] FLAG: --manifest-url-header=""
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695253    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695256    2581 flags.go:64] FLAG: --max-open-files="1000000"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695260    2581 flags.go:64] FLAG: --max-pods="110"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695263    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695266    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695269    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 14 12:58:24.701509 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695272    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695276    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695279    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695283    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695291    2581 flags.go:64] FLAG: --node-status-max-images="50"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695297    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695300    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695303    2581 flags.go:64] FLAG: --pod-cidr=""
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695307    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695312    2581 flags.go:64] FLAG: --pod-manifest-path=""
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695316    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695319    2581 flags.go:64] FLAG: --pods-per-core="0"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695328    2581 flags.go:64] FLAG: --port="10250"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695331    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695334    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a6d5aaa6da2c32c4"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695337    2581 flags.go:64] FLAG: --qos-reserved=""
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695340    2581 flags.go:64] FLAG: --read-only-port="10255"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695343    2581 flags.go:64] FLAG: --register-node="true"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695346    2581 flags.go:64] FLAG: --register-with-taints=""
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695353    2581 flags.go:64] FLAG: --registry-burst="10"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695356    2581 flags.go:64] FLAG: --registry-qps="5"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695359    2581 flags.go:64] FLAG: --reserved-cpus=""
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695362    2581 flags.go:64] FLAG: --reserved-memory=""
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695366    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695369    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 14 12:58:24.702131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695372    2581 flags.go:64] FLAG: --rotate-certificates="false"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695375    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695378    2581 flags.go:64] FLAG: --runonce="false"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695381    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695384    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695387    2581 flags.go:64] FLAG: --seccomp-default="false"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695390    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695393    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695396    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695400    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695404    2581 flags.go:64] FLAG: --storage-driver-password="root"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695407    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695412    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695415    2581 flags.go:64] FLAG: --storage-driver-user="root"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695418    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695421    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695424    2581 flags.go:64] FLAG: --system-cgroups=""
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695427    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695432    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695435    2581 flags.go:64] FLAG: --tls-cert-file=""
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695444    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695450    2581 flags.go:64] FLAG: --tls-min-version=""
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695453    2581 flags.go:64] FLAG: --tls-private-key-file=""
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695456    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695459    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 14 12:58:24.702789 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695462    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695465    2581 flags.go:64] FLAG: --v="2"
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695469    2581 flags.go:64] FLAG: --version="false"
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695473    2581 flags.go:64] FLAG: --vmodule=""
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695476    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.695480    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695620    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695625    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695628    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695630    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695634    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695637    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695640    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695642    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695645    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695647    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695652    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695655    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695657    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695662    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695665    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 12:58:24.703438 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695667    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695670    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695673    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695675    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695678    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695680    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695683    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695686    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695689    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695692    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695695    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695697    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695700    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695702    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695705    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695707    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695710    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695712    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695715    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695717    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 12:58:24.703994 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695720    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695723    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695726    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695728    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695732    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695735    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695737    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695741    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695744    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695747    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695750    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695753    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695756    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695759    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695761    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695764    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695768    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695772    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695775    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 12:58:24.704484 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695778    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695781    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695785    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695787    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695790    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695793    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695796    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695798    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695801    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695804    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695807    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695811    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695814    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695817    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695820    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695823    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695825    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695828    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695831    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 12:58:24.704965 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695834    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695838    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695841    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695844    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695848    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695851    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695854    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695857    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695859    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695862    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695864    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695867    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695870    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695872    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695875    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695878    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695880    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695883    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695885    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 12:58:24.705427 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695888    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 12:58:24.706039 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695891    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 12:58:24.706039 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695893    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 12:58:24.706039 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695896    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 12:58:24.706039 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695898    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 12:58:24.706039 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695901    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 12:58:24.706039 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.695903    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 12:58:24.706039 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.696975    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 12:58:24.706039 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.705280    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 14 12:58:24.706039 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.705298    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 14 12:58:24.706282 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.706161    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 14 12:58:24.706282 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.706168    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 14 12:58:24.706282 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706253    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 12:58:24.706282 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706259    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 12:58:24.706282 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706262    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 12:58:24.706282 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706266    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 12:58:24.706282 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706271    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 12:58:24.706282 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706273    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 12:58:24.706282 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706276    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 12:58:24.706282 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706279    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 12:58:24.706282 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706281    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 12:58:24.706282 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706284    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 12:58:24.706282 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706287    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706290    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706293    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706296    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706298    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706301    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706303    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706307    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706309    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706312    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706314    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706317    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706319    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706322    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706324    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706327    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706329    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706332    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706335    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706343    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 12:58:24.706623 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706346    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706349    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706351    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706354    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706356    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706359    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706362    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706364    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706367    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706369    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706373    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706377    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706381    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706384    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706387    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706390    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706393    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706396    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706398    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706401    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 12:58:24.707110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706404    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706407    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706409    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706412    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706414    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706417    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706420    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706422    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706425    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706427    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706430    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706432    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706441    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706444    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706446    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706449    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706451    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706453    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706456    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706458    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 12:58:24.707683 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706461    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706463    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706466    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706468    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706471    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706474    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706477    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706479    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706482    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706485    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706488    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706490    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706493    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706495    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706498    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706500    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706503    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706505    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706508    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706511    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 12:58:24.708172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706513    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 12:58:24.708674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706515    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 12:58:24.708674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706518    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 12:58:24.708674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706521    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 12:58:24.708674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706523    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 12:58:24.708674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706531    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 12:58:24.708674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706534    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 12:58:24.708674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706538    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 12:58:24.708674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706541    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 12:58:24.708674 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.706547    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 12:58:24.708674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706722    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 12:58:24.708674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706728    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 12:58:24.708674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706731    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 12:58:24.708674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706734    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 12:58:24.708674 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706737    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706740    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706744    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706748    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706751    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706755    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706758    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706761    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706764    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706768    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706770    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706774    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706776    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706779    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706781    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706784    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706786    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706789    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706791    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706794    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 12:58:24.709033 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706796    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706799    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706802    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706805    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706814    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706818    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706821    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706824    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706827    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706830    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706833    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706835    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706837    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706840    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706842    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706845    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706848    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706851    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706854    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 12:58:24.709537 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706856    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706858    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706861    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706864    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706866    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706869    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706871    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706873    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706876    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706879    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706882    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706886    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706888    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706891    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706893    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706896    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706898    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706901    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706909    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 12:58:24.710172 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706912    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706915    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706918    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706920    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706923    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706925    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706928    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706930    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706933    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706935    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706938    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706940    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706943    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706945    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706948    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706950    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706953    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706956    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706958    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706961    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 12:58:24.710671 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706964    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706966    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706969    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706972    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706974    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706976    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706979    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706982    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706984    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706987    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706989    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.706992    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.707001    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.707004    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.707007    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.707009    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 12:58:24.711162 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:24.707012    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 12:58:24.711539 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.707017    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 12:58:24.711539 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.707171    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 14 12:58:24.711539 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.710210    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 14 12:58:24.711539 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.711146    2581 server.go:1019] "Starting client certificate rotation"
Sep 14 12:58:24.711539 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.711246    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 12:58:24.712051 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.712037    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 12:58:24.713656 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.713642    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 12:58:24.714877 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.714860    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 12:58:24.725846 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.725822    2581 log.go:25] "Validated CRI v1 runtime API"
Sep 14 12:58:24.729518 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.729500    2581 log.go:25] "Validated CRI v1 image API"
Sep 14 12:58:24.730427 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.730413    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 14 12:58:24.735394 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.735369    2581 fs.go:135] Filesystem UUIDs: map[706cf6b0-8084-4221-b65d-6b947e45a85e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 eb41d1a1-f0a7-4974-a647-9b4df76c0f47:/dev/nvme0n1p4]
Sep 14 12:58:24.735461 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.735394    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 14 12:58:24.741656 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.741524    2581 manager.go:216] Machine: {Timestamp:2026-09-14 12:58:24.740193626 +0000 UTC m=+0.352480461 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3127538 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec218c50c0c81fb34d765000fecb69a6 SystemUUID:ec218c50-c0c8-1fb3-4d76-5000fecb69a6 BootID:49288f15-8d24-4fe2-b508-64a8fd596767 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e3:84:a1:1b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e3:84:a1:1b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:47:50:7c:09:98 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 14 12:58:24.741656 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.741648    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 14 12:58:24.741824 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.741735    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 14 12:58:24.742096 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.742078    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 12:58:24.742806 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.742777    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 14 12:58:24.742973 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.742815    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-35.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 14 12:58:24.744681 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.744669    2581 topology_manager.go:138] "Creating topology manager with none policy"
Sep 14 12:58:24.744745 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.744686    2581 container_manager_linux.go:309] "Creating device plugin manager"
Sep 14 12:58:24.744745 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.744697    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 12:58:24.745393 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.745379    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 12:58:24.746318 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.746306    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 14 12:58:24.750215 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.750201    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 12:58:24.750345 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.750334    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 14 12:58:24.750398 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.750372    2581 kubelet.go:486] "Attempting to sync node with API server"
Sep 14 12:58:24.750398 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.750385    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 14 12:58:24.750490 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.750404    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 14 12:58:24.750490 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.750420    2581 kubelet.go:398] "Adding apiserver pod source"
Sep 14 12:58:24.750490 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.750433    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 14 12:58:24.750875 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.750861    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 12:58:24.750939 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.750886    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 12:58:24.754408 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.754387    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 14 12:58:24.756555 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.756541    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 14 12:58:24.756632 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.756588    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 14 12:58:24.758068 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.758056    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 14 12:58:24.758104 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.758073    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 14 12:58:24.758104 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.758079    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 14 12:58:24.758104 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.758085    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 14 12:58:24.758104 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.758093    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 14 12:58:24.758104 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.758101    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 14 12:58:24.758237 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.758117    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 14 12:58:24.758237 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.758123    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 14 12:58:24.758237 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.758130    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 14 12:58:24.758237 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.758136    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 14 12:58:24.758237 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.758144    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 14 12:58:24.758237 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.758149    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 14 12:58:24.758970 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.758958    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 14 12:58:24.758970 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.758970    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 14 12:58:24.761305 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:24.761273    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-35.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 14 12:58:24.761468 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:24.761444    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 14 12:58:24.762507 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.762485    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-35.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 14 12:58:24.763272 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.763257    2581 server.go:1268] "Started kubelet"
Sep 14 12:58:24.763382 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.763329    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 14 12:58:24.763464 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.763371    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 14 12:58:24.763559 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.763484    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 14 12:58:24.764301 ip-10-0-135-35 systemd[1]: Started Kubernetes Kubelet.
Sep 14 12:58:24.765288 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.764873    2581 server.go:310] "Adding debug handlers to kubelet server"
Sep 14 12:58:24.765288 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.764931    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 14 12:58:24.770033 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.770010    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8fs28"
Sep 14 12:58:24.770179 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:24.770165    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 14 12:58:24.770350 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:24.769381    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-35.ec2.internal.18d5315d89cc5a69  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-35.ec2.internal,UID:ip-10-0-135-35.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-35.ec2.internal,},FirstTimestamp:2026-09-14 12:58:24.763230825 +0000 UTC m=+0.375517661,LastTimestamp:2026-09-14 12:58:24.763230825 +0000 UTC m=+0.375517661,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-35.ec2.internal,}"
Sep 14 12:58:24.771968 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.771948    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 14 12:58:24.772507 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.772488    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 14 12:58:24.772643 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:24.772615    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-35.ec2.internal\" not found"
Sep 14 12:58:24.773225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.773208    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 14 12:58:24.773225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.773209    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 14 12:58:24.773225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.773236    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 14 12:58:24.773413 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.773394    2581 reconstruct.go:97] "Volume reconstruction finished"
Sep 14 12:58:24.773413 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.773408    2581 reconciler.go:29] "Reconciler: start to sync state"
Sep 14 12:58:24.773570 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.773557    2581 factory.go:55] Registering systemd factory
Sep 14 12:58:24.773624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.773590    2581 factory.go:223] Registration of the systemd container factory successfully
Sep 14 12:58:24.773810 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.773795    2581 factory.go:153] Registering CRI-O factory
Sep 14 12:58:24.773810 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.773809    2581 factory.go:223] Registration of the crio container factory successfully
Sep 14 12:58:24.773903 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.773860    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 14 12:58:24.773903 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.773874    2581 factory.go:103] Registering Raw factory
Sep 14 12:58:24.773903 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.773884    2581 manager.go:1196] Started watching for new ooms in manager
Sep 14 12:58:24.774766 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:24.774737    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 14 12:58:24.774838 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:24.774769    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-35.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 14 12:58:24.774879 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.774860    2581 manager.go:349] Starting recovery of all containers
Sep 14 12:58:24.775553 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.775534    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8fs28"
Sep 14 12:58:24.785777 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.785498    2581 manager.go:354] Recovery completed
Sep 14 12:58:24.790761 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.790545    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 12:58:24.792224 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.792208    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 12:58:24.792274 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.792243    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 12:58:24.792274 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.792254    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasSufficientPID"
Sep 14 12:58:24.792833 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.792816    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 14 12:58:24.792833 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.792826    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 12:58:24.792945 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.792829    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 14 12:58:24.792945 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.792895    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 12:58:24.793478 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.793464    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 12:58:24.793555 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.793487    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 12:58:24.793555 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.793496    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasSufficientPID"
Sep 14 12:58:24.873327 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:24.873297    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-35.ec2.internal\" not found"
Sep 14 12:58:24.938036 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.916851    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 14 12:58:24.938036 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.918109    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 14 12:58:24.938036 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.918184    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 14 12:58:24.938036 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.918223    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 14 12:58:24.938036 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:24.918332    2581 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 14 12:58:24.938036 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.921041    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 12:58:24.938036 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.936734    2581 policy_none.go:49] "None policy: Start"
Sep 14 12:58:24.938036 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.936752    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 14 12:58:24.938036 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.936765    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 14 12:58:24.938803 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.938789    2581 policy_none.go:47] "Start"
Sep 14 12:58:24.964122 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.964106    2581 manager.go:337] "Starting Device Plugin manager"
Sep 14 12:58:24.964208 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:24.964140    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 14 12:58:24.964208 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.964150    2581 server.go:87] "Starting device plugin registration server"
Sep 14 12:58:24.964504 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.964491    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 14 12:58:24.964590 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.964504    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 14 12:58:24.964644 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.964618    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 14 12:58:24.964714 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.964698    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 14 12:58:24.964714 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:24.964711    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 14 12:58:24.966970 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:24.966883    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 14 12:58:24.966970 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:24.966942    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-35.ec2.internal\" not found"
Sep 14 12:58:24.981550 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:24.981528    2581 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-35.ec2.internal\" not found" node="ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.018659 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.018628    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-35.ec2.internal"]
Sep 14 12:58:25.018770 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.018706    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 12:58:25.019662 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.019647    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 12:58:25.019734 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.019675    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 12:58:25.019734 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.019685    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasSufficientPID"
Sep 14 12:58:25.021980 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.021967    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 12:58:25.022094 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.022080    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.022136 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.022111    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 12:58:25.022715 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.022690    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 12:58:25.022715 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.022699    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 12:58:25.022715 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.022714    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 12:58:25.022856 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.022722    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 12:58:25.022856 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.022738    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasSufficientPID"
Sep 14 12:58:25.022856 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.022725    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasSufficientPID"
Sep 14 12:58:25.024972 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.024955    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.025053 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.024981    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 12:58:25.025661 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.025637    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 12:58:25.025765 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.025664    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 12:58:25.025765 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.025679    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasSufficientPID"
Sep 14 12:58:25.041098 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:25.041076    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-35.ec2.internal\" not found" node="ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.044894 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:25.044877    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-35.ec2.internal\" not found" node="ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.065615 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.065598    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 12:58:25.066504 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.066491    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 12:58:25.066561 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.066520    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 12:58:25.066561 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.066530    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeHasSufficientPID"
Sep 14 12:58:25.066561 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.066550    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.075452 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.075436    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.075517 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:25.075457    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-35.ec2.internal\": node \"ip-10-0-135-35.ec2.internal\" not found"
Sep 14 12:58:25.075625 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.075607    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/19233420114c0f8b0b4e91a37e88ffae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal\" (UID: \"19233420114c0f8b0b4e91a37e88ffae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.075686 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.075641    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19233420114c0f8b0b4e91a37e88ffae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal\" (UID: \"19233420114c0f8b0b4e91a37e88ffae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.075686 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.075669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1db45f87cf9fbe960d92124e3bae8a9e-config\") pod \"kube-apiserver-proxy-ip-10-0-135-35.ec2.internal\" (UID: \"1db45f87cf9fbe960d92124e3bae8a9e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.099681 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:25.099659    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-35.ec2.internal\" not found"
Sep 14 12:58:25.176213 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.176186    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/19233420114c0f8b0b4e91a37e88ffae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal\" (UID: \"19233420114c0f8b0b4e91a37e88ffae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.176314 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.176215    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19233420114c0f8b0b4e91a37e88ffae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal\" (UID: \"19233420114c0f8b0b4e91a37e88ffae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.176314 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.176232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1db45f87cf9fbe960d92124e3bae8a9e-config\") pod \"kube-apiserver-proxy-ip-10-0-135-35.ec2.internal\" (UID: \"1db45f87cf9fbe960d92124e3bae8a9e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.176314 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.176307    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1db45f87cf9fbe960d92124e3bae8a9e-config\") pod \"kube-apiserver-proxy-ip-10-0-135-35.ec2.internal\" (UID: \"1db45f87cf9fbe960d92124e3bae8a9e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.176408 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.176310    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19233420114c0f8b0b4e91a37e88ffae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal\" (UID: \"19233420114c0f8b0b4e91a37e88ffae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.176408 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:25.176311    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/19233420114c0f8b0b4e91a37e88ffae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal\" (UID: \"19233420114c0f8b0b4e91a37e88ffae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:25.199936 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:25.199877    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-35.ec2.internal\" not found"
Sep 14 12:58:27.621621 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:27.621583    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-35.ec2.internal\" not found"
Sep 14 12:58:27.664788 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:27.664750    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:27.668739 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:27.668719    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:27.722674 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:27.722642    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-35.ec2.internal\" not found"
Sep 14 12:58:27.823266 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:27.823185    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-35.ec2.internal\" not found"
Sep 14 12:58:27.923697 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:27.923663    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-35.ec2.internal\" not found"
Sep 14 12:58:27.994651 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:27.994625    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 12:58:28.024253 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.024233    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-35.ec2.internal\" not found"
Sep 14 12:58:28.032403 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.032384    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 14 12:58:28.032543 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.032507    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 12:58:28.032607 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.032566    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 12:58:28.058858 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.058821    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 12:58:28.071772 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.071753    2581 apiserver.go:52] "Watching apiserver"
Sep 14 12:58:28.081291 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.081237    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 14 12:58:28.081603 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.081582    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-dszvl","openshift-dns/node-resolver-vftv8","openshift-multus/multus-additional-cni-plugins-zdlhr","openshift-multus/multus-sfhjz","openshift-multus/network-metrics-daemon-49fdc","openshift-network-diagnostics/network-check-target-6p464","openshift-ovn-kubernetes/ovnkube-node-d8mhr","kube-system/konnectivity-agent-9462f","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76","openshift-cluster-node-tuning-operator/tuned-nqllm","openshift-image-registry/node-ca-t797n","openshift-network-operator/iptables-alerter-njh47"]
Sep 14 12:58:28.084020 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.084001    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:28.084097 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.084070    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:28.085918 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.085898    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vftv8"
Sep 14 12:58:28.088090 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.088066    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 14 12:58:28.088187 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.088130    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qrtm6\""
Sep 14 12:58:28.088553 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.088307    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 14 12:58:28.088553 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.088455    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.090785 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.090753    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.090931 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.090914    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 14 12:58:28.091033 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.091013    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8h7cx\""
Sep 14 12:58:28.091271 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.091255    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 14 12:58:28.091712 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.091698    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 14 12:58:28.091772 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.091749    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 14 12:58:28.091837 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.091821    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 14 12:58:28.092852 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.092836    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:28.092939 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.092886    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:28.092991 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.092976    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7ls5v\""
Sep 14 12:58:28.093036 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.093013    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 14 12:58:28.093101 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.093084    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 14 12:58:28.093618 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.093605    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:28.094900 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.094881    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:28.094985 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.094944    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:28.097188 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.097172    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.099300 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.099284    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9462f"
Sep 14 12:58:28.099843 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.099828    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 14 12:58:28.099912 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.099825    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-57lj5\""
Sep 14 12:58:28.099912 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.099861    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 14 12:58:28.099912 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.099864    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 14 12:58:28.099912 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.099876    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 14 12:58:28.100148 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100134    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 14 12:58:28.100186 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100165    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 14 12:58:28.100186 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100172    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/db77a7f7-b9e7-47f8-a7d0-524847920a6b-kubelet-config\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:28.100260 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100190    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/db77a7f7-b9e7-47f8-a7d0-524847920a6b-dbus\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:28.100260 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100207    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpt8m\" (UniqueName: \"kubernetes.io/projected/4cc3d786-1998-4dd8-b68d-739249a7ec18-kube-api-access-kpt8m\") pod \"node-resolver-vftv8\" (UID: \"4cc3d786-1998-4dd8-b68d-739249a7ec18\") " pod="openshift-dns/node-resolver-vftv8"
Sep 14 12:58:28.100260 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100230    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1346f708-a2ee-4552-8a2c-1de1e10b4277-cni-binary-copy\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.100260 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-multus-socket-dir-parent\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.100390 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100273    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-var-lib-cni-multus\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.100390 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100289    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-multus-conf-dir\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.100390 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100327    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j6nd\" (UniqueName: \"kubernetes.io/projected/1346f708-a2ee-4552-8a2c-1de1e10b4277-kube-api-access-7j6nd\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.100390 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100350    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:28.100390 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100369    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b59f7c6d-e692-4bb7-baca-3b4e6c932134-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.100601 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100425    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-var-lib-kubelet\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.100601 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100475    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs\") pod \"network-metrics-daemon-49fdc\" (UID: \"f05d48a3-962c-42c0-aba5-1cd6441c74d9\") " pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:28.100601 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100516    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dmnf\" (UniqueName: \"kubernetes.io/projected/f05d48a3-962c-42c0-aba5-1cd6441c74d9-kube-api-access-5dmnf\") pod \"network-metrics-daemon-49fdc\" (UID: \"f05d48a3-962c-42c0-aba5-1cd6441c74d9\") " pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:28.100601 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100558    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-run-k8s-cni-cncf-io\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.100601 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-etc-kubernetes\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.100601 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100598    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc3d786-1998-4dd8-b68d-739249a7ec18-tmp-dir\") pod \"node-resolver-vftv8\" (UID: \"4cc3d786-1998-4dd8-b68d-739249a7ec18\") " pod="openshift-dns/node-resolver-vftv8"
Sep 14 12:58:28.100820 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100612    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b59f7c6d-e692-4bb7-baca-3b4e6c932134-cnibin\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.100820 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100638    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b59f7c6d-e692-4bb7-baca-3b4e6c932134-cni-binary-copy\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.100820 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100677    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b59f7c6d-e692-4bb7-baca-3b4e6c932134-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.100820 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gw6r\" (UniqueName: \"kubernetes.io/projected/b59f7c6d-e692-4bb7-baca-3b4e6c932134-kube-api-access-6gw6r\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.100820 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100717    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-run-netns\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.100820 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100734    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-hostroot\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.100820 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100748    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-run-multus-certs\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.100820 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mbnm\" (UniqueName: \"kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm\") pod \"network-check-target-6p464\" (UID: \"f9624365-2f56-45f5-ba74-617be3896ac4\") " pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:28.100820 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100776    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-system-cni-dir\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.100820 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100789    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-var-lib-cni-bin\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.100820 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-multus-cni-dir\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.100820 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100819    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1346f708-a2ee-4552-8a2c-1de1e10b4277-multus-daemon-config\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.101147 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc3d786-1998-4dd8-b68d-739249a7ec18-hosts-file\") pod \"node-resolver-vftv8\" (UID: \"4cc3d786-1998-4dd8-b68d-739249a7ec18\") " pod="openshift-dns/node-resolver-vftv8"
Sep 14 12:58:28.101147 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100869    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b59f7c6d-e692-4bb7-baca-3b4e6c932134-system-cni-dir\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.101147 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100891    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b59f7c6d-e692-4bb7-baca-3b4e6c932134-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.101147 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.100914    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-cnibin\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.101256 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.101142    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-13 12:53:27 +0000 UTC" deadline="2028-04-25 04:36:54.207926666 +0000 UTC"
Sep 14 12:58:28.101256 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.101179    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14127h38m26.106749491s"
Sep 14 12:58:28.101453 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.101437    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.103780 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.103766    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.104627 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.104610    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 14 12:58:28.104695 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.104636    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 14 12:58:28.104695 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.104676    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 14 12:58:28.104774 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.104745    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-znsxx\""
Sep 14 12:58:28.104937 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.104922    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 14 12:58:28.104990 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.104951    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 14 12:58:28.105021 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.104998    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-64cc5\""
Sep 14 12:58:28.105935 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.105919    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t797n"
Sep 14 12:58:28.106034 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.105970    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 14 12:58:28.106187 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.106171    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t6ffv\""
Sep 14 12:58:28.106551 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.106516    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 14 12:58:28.108143 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.108124    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal"]
Sep 14 12:58:28.108243 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.108149    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 12:58:28.108243 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.108213    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-njh47"
Sep 14 12:58:28.108349 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.108216    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-35.ec2.internal"
Sep 14 12:58:28.109091 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.109076    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-h8mlx\""
Sep 14 12:58:28.109183 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.109150    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 14 12:58:28.109733 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.109708    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 14 12:58:28.109825 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.109732    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 14 12:58:28.110424 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.110410    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 14 12:58:28.110884 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.110866    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qd26v\""
Sep 14 12:58:28.110967 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.110911    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 14 12:58:28.111097 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.111081    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 14 12:58:28.111178 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.111102    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 12:58:28.119154 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.119139    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-35.ec2.internal"]
Sep 14 12:58:28.119281 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.119271    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 12:58:28.130054 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.130036    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-n6khf"
Sep 14 12:58:28.141831 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.141817    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-n6khf"
Sep 14 12:58:28.194905 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.194889    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 14 12:58:28.201880 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.201860    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b59f7c6d-e692-4bb7-baca-3b4e6c932134-cni-binary-copy\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.201974 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.201886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b59f7c6d-e692-4bb7-baca-3b4e6c932134-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.201974 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.201927    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-run-multus-certs\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.201974 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.201954    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-sysconfig\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.202111 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.201976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-kubernetes\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.202111 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202000    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.202111 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202028    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mbnm\" (UniqueName: \"kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm\") pod \"network-check-target-6p464\" (UID: \"f9624365-2f56-45f5-ba74-617be3896ac4\") " pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:28.202111 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202051    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-etc-openvswitch\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.202111 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202075    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfnd5\" (UniqueName: \"kubernetes.io/projected/9e5cd353-35cb-4693-a0e3-a49a546d8528-kube-api-access-vfnd5\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.202324 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202114    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/16ccf89b-07a3-4902-89ce-4e2ba343dfa8-iptables-alerter-script\") pod \"iptables-alerter-njh47\" (UID: \"16ccf89b-07a3-4902-89ce-4e2ba343dfa8\") " pod="openshift-network-operator/iptables-alerter-njh47"
Sep 14 12:58:28.202324 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-run-multus-certs\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.202324 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202204    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1346f708-a2ee-4552-8a2c-1de1e10b4277-multus-daemon-config\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.202324 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-socket-dir\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.202324 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202260    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b59f7c6d-e692-4bb7-baca-3b4e6c932134-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.202324 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202300    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-kubelet\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.202598 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202332    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/db77a7f7-b9e7-47f8-a7d0-524847920a6b-kubelet-config\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:28.202598 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202367    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-var-lib-cni-multus\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.202598 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202394    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-lib-modules\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.202598 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202418    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxv2d\" (UniqueName: \"kubernetes.io/projected/f6370fdb-7aec-47e0-86fb-707bcd59eb50-kube-api-access-qxv2d\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.202598 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202418    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/db77a7f7-b9e7-47f8-a7d0-524847920a6b-kubelet-config\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:28.202598 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202442    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-run-openvswitch\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.202598 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202467    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e5cd353-35cb-4693-a0e3-a49a546d8528-ovn-node-metrics-cert\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.202598 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202490    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b59f7c6d-e692-4bb7-baca-3b4e6c932134-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.202598 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202496    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-var-lib-cni-multus\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.202598 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202490    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gshfv\" (UniqueName: \"kubernetes.io/projected/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-kube-api-access-gshfv\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.202598 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202567    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b59f7c6d-e692-4bb7-baca-3b4e6c932134-cni-binary-copy\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.202598 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202570    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-var-lib-openvswitch\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202619    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-registration-dir\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202647    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e6b0822-d820-47f4-a652-f56f2f95707e-host\") pod \"node-ca-t797n\" (UID: \"8e6b0822-d820-47f4-a652-f56f2f95707e\") " pod="openshift-image-registry/node-ca-t797n"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8e6b0822-d820-47f4-a652-f56f2f95707e-serviceca\") pod \"node-ca-t797n\" (UID: \"8e6b0822-d820-47f4-a652-f56f2f95707e\") " pod="openshift-image-registry/node-ca-t797n"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202734    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-modprobe-d\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202756    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-var-lib-kubelet\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202760    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1346f708-a2ee-4552-8a2c-1de1e10b4277-multus-daemon-config\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202779    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-sys-fs\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202780    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b59f7c6d-e692-4bb7-baca-3b4e6c932134-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202802    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwcph\" (UniqueName: \"kubernetes.io/projected/8e6b0822-d820-47f4-a652-f56f2f95707e-kube-api-access-gwcph\") pod \"node-ca-t797n\" (UID: \"8e6b0822-d820-47f4-a652-f56f2f95707e\") " pod="openshift-image-registry/node-ca-t797n"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202832    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs\") pod \"network-metrics-daemon-49fdc\" (UID: \"f05d48a3-962c-42c0-aba5-1cd6441c74d9\") " pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-run-k8s-cni-cncf-io\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202864    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16ccf89b-07a3-4902-89ce-4e2ba343dfa8-host-slash\") pod \"iptables-alerter-njh47\" (UID: \"16ccf89b-07a3-4902-89ce-4e2ba343dfa8\") " pod="openshift-network-operator/iptables-alerter-njh47"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202895    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-run-k8s-cni-cncf-io\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202901    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-tuned\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.202918    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.202920    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:28.202984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202923    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc3d786-1998-4dd8-b68d-739249a7ec18-tmp-dir\") pod \"node-resolver-vftv8\" (UID: \"4cc3d786-1998-4dd8-b68d-739249a7ec18\") " pod="openshift-dns/node-resolver-vftv8"
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.202969    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gw6r\" (UniqueName: \"kubernetes.io/projected/b59f7c6d-e692-4bb7-baca-3b4e6c932134-kube-api-access-6gw6r\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.202988    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs podName:f05d48a3-962c-42c0-aba5-1cd6441c74d9 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:28.702958539 +0000 UTC m=+1.994292355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs") pod "network-metrics-daemon-49fdc" (UID: "f05d48a3-962c-42c0-aba5-1cd6441c74d9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-run-netns\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203034    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-hostroot\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203057    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-systemd\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203071    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-run-netns\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203079    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-run\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.203110    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret podName:db77a7f7-b9e7-47f8-a7d0-524847920a6b nodeName:}" failed. No retries permitted until 2026-09-14 12:58:28.703091714 +0000 UTC m=+1.994425513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret") pod "global-pull-secret-syncer-dszvl" (UID: "db77a7f7-b9e7-47f8-a7d0-524847920a6b") : object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203131    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-host\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203137    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-hostroot\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203179    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-system-cni-dir\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203199    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-var-lib-cni-bin\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-slash\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9e5cd353-35cb-4693-a0e3-a49a546d8528-ovnkube-script-lib\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203260    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4-agent-certs\") pod \"konnectivity-agent-9462f\" (UID: \"48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4\") " pod="kube-system/konnectivity-agent-9462f"
Sep 14 12:58:28.203624 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203261    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-var-lib-cni-bin\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203278    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-system-cni-dir\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203291    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4-konnectivity-ca\") pod \"konnectivity-agent-9462f\" (UID: \"48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4\") " pod="kube-system/konnectivity-agent-9462f"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203325    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-multus-cni-dir\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203341    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-systemd-units\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203352    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc3d786-1998-4dd8-b68d-739249a7ec18-tmp-dir\") pod \"node-resolver-vftv8\" (UID: \"4cc3d786-1998-4dd8-b68d-739249a7ec18\") " pod="openshift-dns/node-resolver-vftv8"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203364    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-log-socket\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203374    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-multus-cni-dir\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203391    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-cni-netd\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203408    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-etc-selinux\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203425    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc3d786-1998-4dd8-b68d-739249a7ec18-hosts-file\") pod \"node-resolver-vftv8\" (UID: \"4cc3d786-1998-4dd8-b68d-739249a7ec18\") " pod="openshift-dns/node-resolver-vftv8"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203479    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b59f7c6d-e692-4bb7-baca-3b4e6c932134-system-cni-dir\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203496    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-cnibin\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b59f7c6d-e692-4bb7-baca-3b4e6c932134-system-cni-dir\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203479    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cc3d786-1998-4dd8-b68d-739249a7ec18-hosts-file\") pod \"node-resolver-vftv8\" (UID: \"4cc3d786-1998-4dd8-b68d-739249a7ec18\") " pod="openshift-dns/node-resolver-vftv8"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203510    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-sysctl-d\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-sysctl-conf\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.204218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203585    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6370fdb-7aec-47e0-86fb-707bcd59eb50-tmp\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203600    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-run-systemd\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203615    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9e5cd353-35cb-4693-a0e3-a49a546d8528-ovnkube-config\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-cnibin\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/db77a7f7-b9e7-47f8-a7d0-524847920a6b-dbus\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpt8m\" (UniqueName: \"kubernetes.io/projected/4cc3d786-1998-4dd8-b68d-739249a7ec18-kube-api-access-kpt8m\") pod \"node-resolver-vftv8\" (UID: \"4cc3d786-1998-4dd8-b68d-739249a7ec18\") " pod="openshift-dns/node-resolver-vftv8"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203684    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1346f708-a2ee-4552-8a2c-1de1e10b4277-cni-binary-copy\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-multus-socket-dir-parent\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203711    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-multus-conf-dir\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203729    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j6nd\" (UniqueName: \"kubernetes.io/projected/1346f708-a2ee-4552-8a2c-1de1e10b4277-kube-api-access-7j6nd\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203754    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-run-ovn-kubernetes\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203768    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-multus-socket-dir-parent\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203777    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/db77a7f7-b9e7-47f8-a7d0-524847920a6b-dbus\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-cni-bin\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203784    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-multus-conf-dir\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203819    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-run-netns\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203852    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-run-ovn\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.204721 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203875    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-node-log\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203901    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t92rz\" (UniqueName: \"kubernetes.io/projected/16ccf89b-07a3-4902-89ce-4e2ba343dfa8-kube-api-access-t92rz\") pod \"iptables-alerter-njh47\" (UID: \"16ccf89b-07a3-4902-89ce-4e2ba343dfa8\") " pod="openshift-network-operator/iptables-alerter-njh47"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203926    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b59f7c6d-e692-4bb7-baca-3b4e6c932134-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-var-lib-kubelet\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.203994    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-device-dir\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.204020    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dmnf\" (UniqueName: \"kubernetes.io/projected/f05d48a3-962c-42c0-aba5-1cd6441c74d9-kube-api-access-5dmnf\") pod \"network-metrics-daemon-49fdc\" (UID: \"f05d48a3-962c-42c0-aba5-1cd6441c74d9\") " pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.204044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-etc-kubernetes\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.204058    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b59f7c6d-e692-4bb7-baca-3b4e6c932134-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.204065    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-sys\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.204022    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-host-var-lib-kubelet\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.204089    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9e5cd353-35cb-4693-a0e3-a49a546d8528-env-overrides\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.204101    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1346f708-a2ee-4552-8a2c-1de1e10b4277-etc-kubernetes\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.204115    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b59f7c6d-e692-4bb7-baca-3b4e6c932134-cnibin\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.204120    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1346f708-a2ee-4552-8a2c-1de1e10b4277-cni-binary-copy\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.205225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.204172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b59f7c6d-e692-4bb7-baca-3b4e6c932134-cnibin\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.210567 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.210542    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 12:58:28.210567 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.210562    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 12:58:28.210567 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.210571    2581 projected.go:196] Error preparing data for projected volume kube-api-access-6mbnm for pod openshift-network-diagnostics/network-check-target-6p464: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:28.210722 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.210623    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm podName:f9624365-2f56-45f5-ba74-617be3896ac4 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:28.710612261 +0000 UTC m=+2.001946056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6mbnm" (UniqueName: "kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm") pod "network-check-target-6p464" (UID: "f9624365-2f56-45f5-ba74-617be3896ac4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:28.212482 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.212462    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 14 12:58:28.215738 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.215720    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gw6r\" (UniqueName: \"kubernetes.io/projected/b59f7c6d-e692-4bb7-baca-3b4e6c932134-kube-api-access-6gw6r\") pod \"multus-additional-cni-plugins-zdlhr\" (UID: \"b59f7c6d-e692-4bb7-baca-3b4e6c932134\") " pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.215810 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.215795    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dmnf\" (UniqueName: \"kubernetes.io/projected/f05d48a3-962c-42c0-aba5-1cd6441c74d9-kube-api-access-5dmnf\") pod \"network-metrics-daemon-49fdc\" (UID: \"f05d48a3-962c-42c0-aba5-1cd6441c74d9\") " pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:28.215810 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.215802    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j6nd\" (UniqueName: \"kubernetes.io/projected/1346f708-a2ee-4552-8a2c-1de1e10b4277-kube-api-access-7j6nd\") pod \"multus-sfhjz\" (UID: \"1346f708-a2ee-4552-8a2c-1de1e10b4277\") " pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.215904 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.215801    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpt8m\" (UniqueName: \"kubernetes.io/projected/4cc3d786-1998-4dd8-b68d-739249a7ec18-kube-api-access-kpt8m\") pod \"node-resolver-vftv8\" (UID: \"4cc3d786-1998-4dd8-b68d-739249a7ec18\") " pod="openshift-dns/node-resolver-vftv8"
Sep 14 12:58:28.304733 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.304706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4-konnectivity-ca\") pod \"konnectivity-agent-9462f\" (UID: \"48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4\") " pod="kube-system/konnectivity-agent-9462f"
Sep 14 12:58:28.304733 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.304735    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-systemd-units\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.304920 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.304750    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-log-socket\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.304920 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.304763    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-cni-netd\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.304920 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.304824    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-systemd-units\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.304920 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.304867    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-log-socket\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.304920 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.304898    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-cni-netd\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.304920 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.304913    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-etc-selinux\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.305095 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.304934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-sysctl-d\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.305095 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-sysctl-conf\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.305095 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6370fdb-7aec-47e0-86fb-707bcd59eb50-tmp\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.305095 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305052    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-etc-selinux\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.305095 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-run-systemd\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.305095 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305071    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-sysctl-d\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.305288 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305097    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-run-systemd\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.305288 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9e5cd353-35cb-4693-a0e3-a49a546d8528-ovnkube-config\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.305288 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305132    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-sysctl-conf\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.305288 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305151    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-run-ovn-kubernetes\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.305288 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305200    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-cni-bin\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.305288 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305203    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-run-ovn-kubernetes\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.305288 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305226    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-run-netns\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.305288 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305263    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-run-ovn\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.305288 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305273    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4-konnectivity-ca\") pod \"konnectivity-agent-9462f\" (UID: \"48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4\") " pod="kube-system/konnectivity-agent-9462f"
Sep 14 12:58:28.305288 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305285    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-node-log\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.305750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305243    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-cni-bin\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.305750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305321    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-node-log\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.305750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305322    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t92rz\" (UniqueName: \"kubernetes.io/projected/16ccf89b-07a3-4902-89ce-4e2ba343dfa8-kube-api-access-t92rz\") pod \"iptables-alerter-njh47\" (UID: \"16ccf89b-07a3-4902-89ce-4e2ba343dfa8\") " pod="openshift-network-operator/iptables-alerter-njh47"
Sep 14 12:58:28.305750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305344    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-run-ovn\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.305750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.305750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305409    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.305750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-device-dir\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.305750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305369    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-run-netns\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.305750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-sys\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.305750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305495    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9e5cd353-35cb-4693-a0e3-a49a546d8528-env-overrides\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.305750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305490    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-device-dir\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.305750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305564    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-sysconfig\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.305750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305588    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-kubernetes\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.305750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305626    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-sys\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.306221 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305684    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-sysconfig\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.306221 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305993    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9e5cd353-35cb-4693-a0e3-a49a546d8528-ovnkube-config\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.306221 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.305996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.306221 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306051    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.306221 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306073    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-etc-openvswitch\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.306221 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306100    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-kubernetes\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.306221 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306110    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfnd5\" (UniqueName: \"kubernetes.io/projected/9e5cd353-35cb-4693-a0e3-a49a546d8528-kube-api-access-vfnd5\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.306221 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306161    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/16ccf89b-07a3-4902-89ce-4e2ba343dfa8-iptables-alerter-script\") pod \"iptables-alerter-njh47\" (UID: \"16ccf89b-07a3-4902-89ce-4e2ba343dfa8\") " pod="openshift-network-operator/iptables-alerter-njh47"
Sep 14 12:58:28.306221 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306192    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-socket-dir\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.306221 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-kubelet\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.306543 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306249    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-lib-modules\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.306543 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxv2d\" (UniqueName: \"kubernetes.io/projected/f6370fdb-7aec-47e0-86fb-707bcd59eb50-kube-api-access-qxv2d\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.306543 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306310    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-run-openvswitch\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.306543 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306340    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e5cd353-35cb-4693-a0e3-a49a546d8528-ovn-node-metrics-cert\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.306543 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306361    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9e5cd353-35cb-4693-a0e3-a49a546d8528-env-overrides\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.306543 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306367    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gshfv\" (UniqueName: \"kubernetes.io/projected/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-kube-api-access-gshfv\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.306543 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-etc-openvswitch\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.306772 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306710    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-lib-modules\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.306772 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-var-lib-openvswitch\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.306843 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306792    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-registration-dir\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.306843 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306822    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e6b0822-d820-47f4-a652-f56f2f95707e-host\") pod \"node-ca-t797n\" (UID: \"8e6b0822-d820-47f4-a652-f56f2f95707e\") " pod="openshift-image-registry/node-ca-t797n"
Sep 14 12:58:28.306950 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306846    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8e6b0822-d820-47f4-a652-f56f2f95707e-serviceca\") pod \"node-ca-t797n\" (UID: \"8e6b0822-d820-47f4-a652-f56f2f95707e\") " pod="openshift-image-registry/node-ca-t797n"
Sep 14 12:58:28.306950 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306898    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-modprobe-d\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.306950 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306926    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-var-lib-kubelet\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.307097 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306957    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-sys-fs\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.307097 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306965    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-socket-dir\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.307097 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.306983    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwcph\" (UniqueName: \"kubernetes.io/projected/8e6b0822-d820-47f4-a652-f56f2f95707e-kube-api-access-gwcph\") pod \"node-ca-t797n\" (UID: \"8e6b0822-d820-47f4-a652-f56f2f95707e\") " pod="openshift-image-registry/node-ca-t797n"
Sep 14 12:58:28.307097 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307050    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16ccf89b-07a3-4902-89ce-4e2ba343dfa8-host-slash\") pod \"iptables-alerter-njh47\" (UID: \"16ccf89b-07a3-4902-89ce-4e2ba343dfa8\") " pod="openshift-network-operator/iptables-alerter-njh47"
Sep 14 12:58:28.307292 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307125    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-tuned\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.307292 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307207    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-systemd\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.307397 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307307    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-modprobe-d\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.307397 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307349    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/16ccf89b-07a3-4902-89ce-4e2ba343dfa8-iptables-alerter-script\") pod \"iptables-alerter-njh47\" (UID: \"16ccf89b-07a3-4902-89ce-4e2ba343dfa8\") " pod="openshift-network-operator/iptables-alerter-njh47"
Sep 14 12:58:28.307397 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307253    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-run\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.307397 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307371    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-var-lib-kubelet\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.307615 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307450    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-run\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.307615 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307490    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-registration-dir\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.307615 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307514    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-sys-fs\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.307615 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307573    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-host\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.307847 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307826    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-systemd\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.307908 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307883    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-kubelet\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.307971 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307919    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-slash\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.307971 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307932    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16ccf89b-07a3-4902-89ce-4e2ba343dfa8-host-slash\") pod \"iptables-alerter-njh47\" (UID: \"16ccf89b-07a3-4902-89ce-4e2ba343dfa8\") " pod="openshift-network-operator/iptables-alerter-njh47"
Sep 14 12:58:28.307971 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307044    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-var-lib-openvswitch\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.308121 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307981    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-host-slash\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.308121 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.307982    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e5cd353-35cb-4693-a0e3-a49a546d8528-run-openvswitch\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.308121 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.308021    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9e5cd353-35cb-4693-a0e3-a49a546d8528-ovnkube-script-lib\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.308121 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.308067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4-agent-certs\") pod \"konnectivity-agent-9462f\" (UID: \"48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4\") " pod="kube-system/konnectivity-agent-9462f"
Sep 14 12:58:28.308321 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.308148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8e6b0822-d820-47f4-a652-f56f2f95707e-serviceca\") pod \"node-ca-t797n\" (UID: \"8e6b0822-d820-47f4-a652-f56f2f95707e\") " pod="openshift-image-registry/node-ca-t797n"
Sep 14 12:58:28.308321 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.308068    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6370fdb-7aec-47e0-86fb-707bcd59eb50-host\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.308421 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.308366    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6370fdb-7aec-47e0-86fb-707bcd59eb50-tmp\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.309192 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.308583    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9e5cd353-35cb-4693-a0e3-a49a546d8528-ovnkube-script-lib\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.310171 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.309373    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e5cd353-35cb-4693-a0e3-a49a546d8528-ovn-node-metrics-cert\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.310275 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.310229    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e6b0822-d820-47f4-a652-f56f2f95707e-host\") pod \"node-ca-t797n\" (UID: \"8e6b0822-d820-47f4-a652-f56f2f95707e\") " pod="openshift-image-registry/node-ca-t797n"
Sep 14 12:58:28.311184 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.311161    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4-agent-certs\") pod \"konnectivity-agent-9462f\" (UID: \"48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4\") " pod="kube-system/konnectivity-agent-9462f"
Sep 14 12:58:28.311787 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.311770    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f6370fdb-7aec-47e0-86fb-707bcd59eb50-etc-tuned\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.322580 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.322553    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfnd5\" (UniqueName: \"kubernetes.io/projected/9e5cd353-35cb-4693-a0e3-a49a546d8528-kube-api-access-vfnd5\") pod \"ovnkube-node-d8mhr\" (UID: \"9e5cd353-35cb-4693-a0e3-a49a546d8528\") " pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.322675 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.322605    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxv2d\" (UniqueName: \"kubernetes.io/projected/f6370fdb-7aec-47e0-86fb-707bcd59eb50-kube-api-access-qxv2d\") pod \"tuned-nqllm\" (UID: \"f6370fdb-7aec-47e0-86fb-707bcd59eb50\") " pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.322675 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.322635    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t92rz\" (UniqueName: \"kubernetes.io/projected/16ccf89b-07a3-4902-89ce-4e2ba343dfa8-kube-api-access-t92rz\") pod \"iptables-alerter-njh47\" (UID: \"16ccf89b-07a3-4902-89ce-4e2ba343dfa8\") " pod="openshift-network-operator/iptables-alerter-njh47"
Sep 14 12:58:28.323221 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.323204    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gshfv\" (UniqueName: \"kubernetes.io/projected/faf54388-f004-49f6-a2e0-3b7bf0d88ab6-kube-api-access-gshfv\") pod \"aws-ebs-csi-driver-node-vjk76\" (UID: \"faf54388-f004-49f6-a2e0-3b7bf0d88ab6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.324420 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.324405    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwcph\" (UniqueName: \"kubernetes.io/projected/8e6b0822-d820-47f4-a652-f56f2f95707e-kube-api-access-gwcph\") pod \"node-ca-t797n\" (UID: \"8e6b0822-d820-47f4-a652-f56f2f95707e\") " pod="openshift-image-registry/node-ca-t797n"
Sep 14 12:58:28.400740 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:28.400695    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19233420114c0f8b0b4e91a37e88ffae.slice/crio-be4fff9053925e67c51ace28678d9dba5403a0829feea429fccb2728065d553b WatchSource:0}: Error finding container be4fff9053925e67c51ace28678d9dba5403a0829feea429fccb2728065d553b: Status 404 returned error can't find the container with id be4fff9053925e67c51ace28678d9dba5403a0829feea429fccb2728065d553b
Sep 14 12:58:28.401283 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:28.401264    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1db45f87cf9fbe960d92124e3bae8a9e.slice/crio-7bd9e567b94deeda3b4b997011c3eaf6435ab8a4569066697e2ffcb83a20c4ed WatchSource:0}: Error finding container 7bd9e567b94deeda3b4b997011c3eaf6435ab8a4569066697e2ffcb83a20c4ed: Status 404 returned error can't find the container with id 7bd9e567b94deeda3b4b997011c3eaf6435ab8a4569066697e2ffcb83a20c4ed
Sep 14 12:58:28.405030 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.405013    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 12:58:28.433493 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.433470    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vftv8"
Sep 14 12:58:28.439427 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.439406    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zdlhr"
Sep 14 12:58:28.442516 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:28.442495    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc3d786_1998_4dd8_b68d_739249a7ec18.slice/crio-57d41c1016abac51b78a591d41451a2b13688123838a0ee1154dbff541b91ff2 WatchSource:0}: Error finding container 57d41c1016abac51b78a591d41451a2b13688123838a0ee1154dbff541b91ff2: Status 404 returned error can't find the container with id 57d41c1016abac51b78a591d41451a2b13688123838a0ee1154dbff541b91ff2
Sep 14 12:58:28.447613 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:28.447592    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb59f7c6d_e692_4bb7_baca_3b4e6c932134.slice/crio-5968d5250e7443cdcde37a53ebbe11dd4e837027ad0360c9291d0cbe593c09d8 WatchSource:0}: Error finding container 5968d5250e7443cdcde37a53ebbe11dd4e837027ad0360c9291d0cbe593c09d8: Status 404 returned error can't find the container with id 5968d5250e7443cdcde37a53ebbe11dd4e837027ad0360c9291d0cbe593c09d8
Sep 14 12:58:28.453723 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.453703    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sfhjz"
Sep 14 12:58:28.455889 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.455868    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:28.463628 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:28.463463    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1346f708_a2ee_4552_8a2c_1de1e10b4277.slice/crio-92eadc15f959c8bd512ba0abe14ab45555b1af83df45e5e423de564a78a3caea WatchSource:0}: Error finding container 92eadc15f959c8bd512ba0abe14ab45555b1af83df45e5e423de564a78a3caea: Status 404 returned error can't find the container with id 92eadc15f959c8bd512ba0abe14ab45555b1af83df45e5e423de564a78a3caea
Sep 14 12:58:28.465488 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:28.465471    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e5cd353_35cb_4693_a0e3_a49a546d8528.slice/crio-540add9740d05cc3896fb98b10add35698e04bf13991e6dcf4959a3b6b641576 WatchSource:0}: Error finding container 540add9740d05cc3896fb98b10add35698e04bf13991e6dcf4959a3b6b641576: Status 404 returned error can't find the container with id 540add9740d05cc3896fb98b10add35698e04bf13991e6dcf4959a3b6b641576
Sep 14 12:58:28.479055 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.479030    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9462f"
Sep 14 12:58:28.488016 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:28.487990    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48b049b1_8dce_45eb_b6e0_2ce78ec5b7b4.slice/crio-c22de3f4bc599839058c386835510a9b6a118b9bd0010fb753f299c2538070ed WatchSource:0}: Error finding container c22de3f4bc599839058c386835510a9b6a118b9bd0010fb753f299c2538070ed: Status 404 returned error can't find the container with id c22de3f4bc599839058c386835510a9b6a118b9bd0010fb753f299c2538070ed
Sep 14 12:58:28.491678 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.491663    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 12:58:28.493788 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.493772    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76"
Sep 14 12:58:28.501352 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:28.501334    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf54388_f004_49f6_a2e0_3b7bf0d88ab6.slice/crio-76ec967a63baf360670f0be7d6218bf62f40d663258b8605f0c92b20dc77955f WatchSource:0}: Error finding container 76ec967a63baf360670f0be7d6218bf62f40d663258b8605f0c92b20dc77955f: Status 404 returned error can't find the container with id 76ec967a63baf360670f0be7d6218bf62f40d663258b8605f0c92b20dc77955f
Sep 14 12:58:28.509493 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.509480    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nqllm"
Sep 14 12:58:28.517160 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:28.517142    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6370fdb_7aec_47e0_86fb_707bcd59eb50.slice/crio-7f92267dc7acccd2e2af3c1bc67fd0ab8bf00ae6317e7012e166526ea890c2b0 WatchSource:0}: Error finding container 7f92267dc7acccd2e2af3c1bc67fd0ab8bf00ae6317e7012e166526ea890c2b0: Status 404 returned error can't find the container with id 7f92267dc7acccd2e2af3c1bc67fd0ab8bf00ae6317e7012e166526ea890c2b0
Sep 14 12:58:28.525312 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.525297    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t797n"
Sep 14 12:58:28.529853 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.529833    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-njh47"
Sep 14 12:58:28.533893 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:28.533870    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e6b0822_d820_47f4_a652_f56f2f95707e.slice/crio-9b385516226ffabe85e255df4aedd98241676fc1029a794ad8698ee150131c3b WatchSource:0}: Error finding container 9b385516226ffabe85e255df4aedd98241676fc1029a794ad8698ee150131c3b: Status 404 returned error can't find the container with id 9b385516226ffabe85e255df4aedd98241676fc1029a794ad8698ee150131c3b
Sep 14 12:58:28.539160 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:58:28.539141    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ccf89b_07a3_4902_89ce_4e2ba343dfa8.slice/crio-2eaf9b99730813c9649148a3bfa3454f7725eff3041662723f6dcb8016cc8b71 WatchSource:0}: Error finding container 2eaf9b99730813c9649148a3bfa3454f7725eff3041662723f6dcb8016cc8b71: Status 404 returned error can't find the container with id 2eaf9b99730813c9649148a3bfa3454f7725eff3041662723f6dcb8016cc8b71
Sep 14 12:58:28.540669 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.540650    2581 kuberuntime_manager.go:1459] "Unhandled Error" err="container iptables-alerter start failed in pod iptables-alerter-njh47_openshift-network-operator(16ccf89b-07a3-4902-89ce-4e2ba343dfa8): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 14 12:58:28.540725 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.540686    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-njh47" podUID="16ccf89b-07a3-4902-89ce-4e2ba343dfa8"
Sep 14 12:58:28.710460 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.710371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:28.710460 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.710408    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs\") pod \"network-metrics-daemon-49fdc\" (UID: \"f05d48a3-962c-42c0-aba5-1cd6441c74d9\") " pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:28.710700 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.710544    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:28.710700 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.710609    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret podName:db77a7f7-b9e7-47f8-a7d0-524847920a6b nodeName:}" failed. No retries permitted until 2026-09-14 12:58:29.710592455 +0000 UTC m=+3.001926249 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret") pod "global-pull-secret-syncer-dszvl" (UID: "db77a7f7-b9e7-47f8-a7d0-524847920a6b") : object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:28.710700 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.710617    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:28.710700 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.710676    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs podName:f05d48a3-962c-42c0-aba5-1cd6441c74d9 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:29.710659393 +0000 UTC m=+3.001993207 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs") pod "network-metrics-daemon-49fdc" (UID: "f05d48a3-962c-42c0-aba5-1cd6441c74d9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:28.810848 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:28.810790    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mbnm\" (UniqueName: \"kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm\") pod \"network-check-target-6p464\" (UID: \"f9624365-2f56-45f5-ba74-617be3896ac4\") " pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:28.811032 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.810966    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 12:58:28.811032 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.810994    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 12:58:28.811032 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.811005    2581 projected.go:196] Error preparing data for projected volume kube-api-access-6mbnm for pod openshift-network-diagnostics/network-check-target-6p464: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:28.811208 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:28.811086    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm podName:f9624365-2f56-45f5-ba74-617be3896ac4 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:29.811066925 +0000 UTC m=+3.102400726 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6mbnm" (UniqueName: "kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm") pod "network-check-target-6p464" (UID: "f9624365-2f56-45f5-ba74-617be3896ac4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:29.145886 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.145844    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 12:53:28 +0000 UTC" deadline="2028-05-31 21:13:41.810926906 +0000 UTC"
Sep 14 12:58:29.145886 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.145882    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15008h15m12.665049352s"
Sep 14 12:58:29.243070 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.239952    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:29.243070 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:29.240095    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:29.243070 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.240310    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:29.243070 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:29.240397    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:29.257182 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.257090    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nqllm" event={"ID":"f6370fdb-7aec-47e0-86fb-707bcd59eb50","Type":"ContainerStarted","Data":"7f92267dc7acccd2e2af3c1bc67fd0ab8bf00ae6317e7012e166526ea890c2b0"}
Sep 14 12:58:29.265015 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.264984    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76" event={"ID":"faf54388-f004-49f6-a2e0-3b7bf0d88ab6","Type":"ContainerStarted","Data":"76ec967a63baf360670f0be7d6218bf62f40d663258b8605f0c92b20dc77955f"}
Sep 14 12:58:29.275861 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.275831    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr" event={"ID":"9e5cd353-35cb-4693-a0e3-a49a546d8528","Type":"ContainerStarted","Data":"540add9740d05cc3896fb98b10add35698e04bf13991e6dcf4959a3b6b641576"}
Sep 14 12:58:29.279073 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.279045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdlhr" event={"ID":"b59f7c6d-e692-4bb7-baca-3b4e6c932134","Type":"ContainerStarted","Data":"5968d5250e7443cdcde37a53ebbe11dd4e837027ad0360c9291d0cbe593c09d8"}
Sep 14 12:58:29.287102 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.287013    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vftv8" event={"ID":"4cc3d786-1998-4dd8-b68d-739249a7ec18","Type":"ContainerStarted","Data":"57d41c1016abac51b78a591d41451a2b13688123838a0ee1154dbff541b91ff2"}
Sep 14 12:58:29.295835 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.295744    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal" event={"ID":"19233420114c0f8b0b4e91a37e88ffae","Type":"ContainerStarted","Data":"be4fff9053925e67c51ace28678d9dba5403a0829feea429fccb2728065d553b"}
Sep 14 12:58:29.307166 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.307136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-njh47" event={"ID":"16ccf89b-07a3-4902-89ce-4e2ba343dfa8","Type":"ContainerStarted","Data":"2eaf9b99730813c9649148a3bfa3454f7725eff3041662723f6dcb8016cc8b71"}
Sep 14 12:58:29.313174 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:29.313143    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0444a9580f430c7c6b52f78667a80a82198c825805f1f2004fdfdf9aed2b95f5\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-njh47" podUID="16ccf89b-07a3-4902-89ce-4e2ba343dfa8"
Sep 14 12:58:29.316307 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.316229    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t797n" event={"ID":"8e6b0822-d820-47f4-a652-f56f2f95707e","Type":"ContainerStarted","Data":"9b385516226ffabe85e255df4aedd98241676fc1029a794ad8698ee150131c3b"}
Sep 14 12:58:29.338958 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.338910    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 12:58:29.343948 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.343917    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9462f" event={"ID":"48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4","Type":"ContainerStarted","Data":"c22de3f4bc599839058c386835510a9b6a118b9bd0010fb753f299c2538070ed"}
Sep 14 12:58:29.360967 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.360930    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sfhjz" event={"ID":"1346f708-a2ee-4552-8a2c-1de1e10b4277","Type":"ContainerStarted","Data":"92eadc15f959c8bd512ba0abe14ab45555b1af83df45e5e423de564a78a3caea"}
Sep 14 12:58:29.377235 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.377014    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-35.ec2.internal" event={"ID":"1db45f87cf9fbe960d92124e3bae8a9e","Type":"ContainerStarted","Data":"7bd9e567b94deeda3b4b997011c3eaf6435ab8a4569066697e2ffcb83a20c4ed"}
Sep 14 12:58:29.718801 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.718765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:29.719019 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.718818    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs\") pod \"network-metrics-daemon-49fdc\" (UID: \"f05d48a3-962c-42c0-aba5-1cd6441c74d9\") " pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:29.719019 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:29.718944    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:29.719019 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:29.719006    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs podName:f05d48a3-962c-42c0-aba5-1cd6441c74d9 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:31.718987126 +0000 UTC m=+5.010320936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs") pod "network-metrics-daemon-49fdc" (UID: "f05d48a3-962c-42c0-aba5-1cd6441c74d9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:29.719401 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:29.719383    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:29.719495 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:29.719434    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret podName:db77a7f7-b9e7-47f8-a7d0-524847920a6b nodeName:}" failed. No retries permitted until 2026-09-14 12:58:31.71942024 +0000 UTC m=+5.010754035 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret") pod "global-pull-secret-syncer-dszvl" (UID: "db77a7f7-b9e7-47f8-a7d0-524847920a6b") : object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:29.819813 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:29.819774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mbnm\" (UniqueName: \"kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm\") pod \"network-check-target-6p464\" (UID: \"f9624365-2f56-45f5-ba74-617be3896ac4\") " pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:29.819987 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:29.819929    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 12:58:29.819987 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:29.819949    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 12:58:29.819987 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:29.819961    2581 projected.go:196] Error preparing data for projected volume kube-api-access-6mbnm for pod openshift-network-diagnostics/network-check-target-6p464: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:29.820152 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:29.820021    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm podName:f9624365-2f56-45f5-ba74-617be3896ac4 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:31.820003956 +0000 UTC m=+5.111337757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6mbnm" (UniqueName: "kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm") pod "network-check-target-6p464" (UID: "f9624365-2f56-45f5-ba74-617be3896ac4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:30.062501 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:30.062422    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 12:58:30.146359 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:30.146315    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 12:53:28 +0000 UTC" deadline="2028-04-07 07:36:41.778705542 +0000 UTC"
Sep 14 12:58:30.146359 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:30.146356    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13698h38m11.632353521s"
Sep 14 12:58:30.239900 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:30.239870    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:30.240074 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:30.240011    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:31.239787 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:31.239750    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:31.240249 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:31.239884    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:31.240249 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:31.239913    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:31.240249 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:31.240027    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:31.740229 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:31.740191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:31.740408 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:31.740243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs\") pod \"network-metrics-daemon-49fdc\" (UID: \"f05d48a3-962c-42c0-aba5-1cd6441c74d9\") " pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:31.740408 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:31.740350    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:31.740408 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:31.740383    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:31.740585 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:31.740422    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret podName:db77a7f7-b9e7-47f8-a7d0-524847920a6b nodeName:}" failed. No retries permitted until 2026-09-14 12:58:35.740402384 +0000 UTC m=+9.031736182 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret") pod "global-pull-secret-syncer-dszvl" (UID: "db77a7f7-b9e7-47f8-a7d0-524847920a6b") : object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:31.740585 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:31.740444    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs podName:f05d48a3-962c-42c0-aba5-1cd6441c74d9 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:35.740432924 +0000 UTC m=+9.031766725 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs") pod "network-metrics-daemon-49fdc" (UID: "f05d48a3-962c-42c0-aba5-1cd6441c74d9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:31.841851 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:31.841741    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mbnm\" (UniqueName: \"kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm\") pod \"network-check-target-6p464\" (UID: \"f9624365-2f56-45f5-ba74-617be3896ac4\") " pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:31.842063 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:31.841940    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 12:58:31.842063 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:31.841967    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 12:58:31.842063 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:31.841983    2581 projected.go:196] Error preparing data for projected volume kube-api-access-6mbnm for pod openshift-network-diagnostics/network-check-target-6p464: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:31.842249 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:31.842093    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm podName:f9624365-2f56-45f5-ba74-617be3896ac4 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:35.842059329 +0000 UTC m=+9.133393124 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6mbnm" (UniqueName: "kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm") pod "network-check-target-6p464" (UID: "f9624365-2f56-45f5-ba74-617be3896ac4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:32.240179 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:32.240142    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:32.240850 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:32.240289    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:33.239438 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:33.239403    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:33.239666 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:33.239568    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:33.239960 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:33.239938    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:33.240077 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:33.240043    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:34.239983 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:34.239946    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:34.240482 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:34.240076    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:35.244329 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:35.243977    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:35.244329 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:35.244106    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:35.244329 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:35.244203    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:35.244329 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:35.244315    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:35.778554 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:35.777965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs\") pod \"network-metrics-daemon-49fdc\" (UID: \"f05d48a3-962c-42c0-aba5-1cd6441c74d9\") " pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:35.778554 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:35.778070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:35.778554 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:35.778200    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:35.778554 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:35.778263    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret podName:db77a7f7-b9e7-47f8-a7d0-524847920a6b nodeName:}" failed. No retries permitted until 2026-09-14 12:58:43.778244481 +0000 UTC m=+17.069578299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret") pod "global-pull-secret-syncer-dszvl" (UID: "db77a7f7-b9e7-47f8-a7d0-524847920a6b") : object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:35.778895 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:35.778609    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:35.778895 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:35.778680    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs podName:f05d48a3-962c-42c0-aba5-1cd6441c74d9 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:43.778661569 +0000 UTC m=+17.069995380 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs") pod "network-metrics-daemon-49fdc" (UID: "f05d48a3-962c-42c0-aba5-1cd6441c74d9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:35.879289 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:35.878989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mbnm\" (UniqueName: \"kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm\") pod \"network-check-target-6p464\" (UID: \"f9624365-2f56-45f5-ba74-617be3896ac4\") " pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:35.879289 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:35.879194    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 12:58:35.879289 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:35.879215    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 12:58:35.879289 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:35.879228    2581 projected.go:196] Error preparing data for projected volume kube-api-access-6mbnm for pod openshift-network-diagnostics/network-check-target-6p464: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:35.879289 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:35.879294    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm podName:f9624365-2f56-45f5-ba74-617be3896ac4 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:43.879277009 +0000 UTC m=+17.170610827 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6mbnm" (UniqueName: "kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm") pod "network-check-target-6p464" (UID: "f9624365-2f56-45f5-ba74-617be3896ac4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:36.239511 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:36.239471    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:36.239705 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:36.239619    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:37.240384 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:37.240351    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:37.240869 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:37.240464    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:37.240869 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:37.240844    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:37.240985 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:37.240947    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:38.240312 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:38.240276    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:38.240481 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:38.240395    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:39.240124 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:39.240086    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:39.240329 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:39.240223    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:39.240329 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:39.240271    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:39.240460 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:39.240361    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:40.239543 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:40.239501    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:40.240038 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:40.239649    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:41.239546 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:41.239498    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:41.239987 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:41.239758    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:41.239987 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:41.239821    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:41.239987 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:41.239934    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:42.240304 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:42.240268    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:42.240722 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:42.240393    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:43.239908 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:43.239876    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:43.240087 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:43.239890    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:43.240087 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:43.240007    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:43.240210 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:43.240082    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:43.841416 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:43.841374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:43.841416 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:43.841426    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs\") pod \"network-metrics-daemon-49fdc\" (UID: \"f05d48a3-962c-42c0-aba5-1cd6441c74d9\") " pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:43.841977 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:43.841555    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:43.841977 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:43.841621    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs podName:f05d48a3-962c-42c0-aba5-1cd6441c74d9 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:59.841603215 +0000 UTC m=+33.132937027 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs") pod "network-metrics-daemon-49fdc" (UID: "f05d48a3-962c-42c0-aba5-1cd6441c74d9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:43.841977 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:43.841556    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:43.842132 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:43.841995    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret podName:db77a7f7-b9e7-47f8-a7d0-524847920a6b nodeName:}" failed. No retries permitted until 2026-09-14 12:58:59.841974165 +0000 UTC m=+33.133307977 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret") pod "global-pull-secret-syncer-dszvl" (UID: "db77a7f7-b9e7-47f8-a7d0-524847920a6b") : object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:43.941876 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:43.941830    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mbnm\" (UniqueName: \"kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm\") pod \"network-check-target-6p464\" (UID: \"f9624365-2f56-45f5-ba74-617be3896ac4\") " pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:43.942043 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:43.942015    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 12:58:43.942043 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:43.942042    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 12:58:43.942134 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:43.942055    2581 projected.go:196] Error preparing data for projected volume kube-api-access-6mbnm for pod openshift-network-diagnostics/network-check-target-6p464: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:43.942134 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:43.942125    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm podName:f9624365-2f56-45f5-ba74-617be3896ac4 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:59.942103514 +0000 UTC m=+33.233437331 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6mbnm" (UniqueName: "kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm") pod "network-check-target-6p464" (UID: "f9624365-2f56-45f5-ba74-617be3896ac4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:44.240167 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:44.240134    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:44.240426 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:44.240262    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:45.239947 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:45.239904    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:45.240444 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:45.239954    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:45.240444 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:45.240067    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:45.240444 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:45.240197    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:46.240216 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:46.240171    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:46.240707 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:46.240312    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:47.240404 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:47.240234    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:47.240912 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:47.240304    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:47.240912 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:47.240465    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:47.240912 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:47.240640    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:47.425126 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:47.425093    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-35.ec2.internal" event={"ID":"1db45f87cf9fbe960d92124e3bae8a9e","Type":"ContainerStarted","Data":"d1a612b42edea484b1681b40e6e79911d1a7da766c2bf804cfac50663936a354"}
Sep 14 12:58:47.427973 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:47.427941    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nqllm" event={"ID":"f6370fdb-7aec-47e0-86fb-707bcd59eb50","Type":"ContainerStarted","Data":"f588b5c3d09411006c81cf42419d900322980b77ec0d91f0b7b2c4f6a4f63f37"}
Sep 14 12:58:47.431841 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:47.431822    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 12:58:47.432205 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:47.432178    2581 generic.go:356] "Generic (PLEG): container finished" podID="9e5cd353-35cb-4693-a0e3-a49a546d8528" containerID="c9ed56d5ece1a0588a574de39eae1eed1ee0c9749f079f1d4591b3dcd7a13cb9" exitCode=1
Sep 14 12:58:47.432293 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:47.432224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr" event={"ID":"9e5cd353-35cb-4693-a0e3-a49a546d8528","Type":"ContainerStarted","Data":"4291f5253cf337de7598d99f53c2e3f3195b2fa53b4fa18ebd24bdc30fc03034"}
Sep 14 12:58:47.432348 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:47.432300    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr" event={"ID":"9e5cd353-35cb-4693-a0e3-a49a546d8528","Type":"ContainerStarted","Data":"6e676c972b8394a76510e2549c4198ee8f1857a57e9813681979129ebed7ae8f"}
Sep 14 12:58:47.432406 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:47.432353    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr" event={"ID":"9e5cd353-35cb-4693-a0e3-a49a546d8528","Type":"ContainerStarted","Data":"22b52f8793fe5e2a4612e33da699f63a161fb0181aeed59c3e55f06c31fac312"}
Sep 14 12:58:47.432406 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:47.432401    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr" event={"ID":"9e5cd353-35cb-4693-a0e3-a49a546d8528","Type":"ContainerDied","Data":"c9ed56d5ece1a0588a574de39eae1eed1ee0c9749f079f1d4591b3dcd7a13cb9"}
Sep 14 12:58:47.432501 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:47.432453    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr" event={"ID":"9e5cd353-35cb-4693-a0e3-a49a546d8528","Type":"ContainerStarted","Data":"8bd1453bfc1b6e21d886b0cd230d4f8be753c239a2427be2a45569ceaa44e163"}
Sep 14 12:58:47.453162 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:47.453117    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-35.ec2.internal" podStartSLOduration=19.453101963 podStartE2EDuration="19.453101963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:47.452714693 +0000 UTC m=+20.744048512" watchObservedRunningTime="2026-09-14 12:58:47.453101963 +0000 UTC m=+20.744435780"
Sep 14 12:58:47.472363 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:47.472165    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nqllm" podStartSLOduration=2.374683331 podStartE2EDuration="20.472148456s" totalImagesPullingTime="18.097465125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:47.47191281 +0000 UTC m=+20.763246618" watchObservedRunningTime="2026-09-14 12:58:47.472148456 +0000 UTC m=+20.763482272"
Sep 14 12:58:48.239870 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.239833    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:48.240066 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:48.239978    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:48.436200 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.436112    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-njh47" event={"ID":"16ccf89b-07a3-4902-89ce-4e2ba343dfa8","Type":"ContainerStarted","Data":"096dbdc33d8b4837abc8d81c206e0ac3cd46d68e7326aab43ab669e62e46b896"}
Sep 14 12:58:48.437825 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.437795    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t797n" event={"ID":"8e6b0822-d820-47f4-a652-f56f2f95707e","Type":"ContainerStarted","Data":"4d00144ead8b7574865bae9545ce7ec663938e5b5abeea17e1e2d2b4e3c7842b"}
Sep 14 12:58:48.439358 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.439332    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9462f" event={"ID":"48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4","Type":"ContainerStarted","Data":"ec3c2ed6bf4949efefdf7365c1e2deeb04a7d3b9d5969a3fed8377a2e2c994d4"}
Sep 14 12:58:48.441039 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.441009    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sfhjz" event={"ID":"1346f708-a2ee-4552-8a2c-1de1e10b4277","Type":"ContainerStarted","Data":"856c097d3411eefec2ce96d4aac9be1fe9fcbb62018f2f4bca93b181fad87d7a"}
Sep 14 12:58:48.442670 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.442614    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76" event={"ID":"faf54388-f004-49f6-a2e0-3b7bf0d88ab6","Type":"ContainerStarted","Data":"71dce06643d6bd6d1dd87f8ec7e343126eede72c2bf088899e3e0ca06172e897"}
Sep 14 12:58:48.445793 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.445773    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 12:58:48.446216 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.446192    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr" event={"ID":"9e5cd353-35cb-4693-a0e3-a49a546d8528","Type":"ContainerStarted","Data":"83dee1284beeaf48f875ed4e22e02bace39541113a923402c65466f31d482dbe"}
Sep 14 12:58:48.447817 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.447790    2581 generic.go:356] "Generic (PLEG): container finished" podID="b59f7c6d-e692-4bb7-baca-3b4e6c932134" containerID="3b33732bdc451478a1ea8b1348f0ff240b20443a108bc9d0facd7b551daea2d2" exitCode=0
Sep 14 12:58:48.447919 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.447866    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdlhr" event={"ID":"b59f7c6d-e692-4bb7-baca-3b4e6c932134","Type":"ContainerDied","Data":"3b33732bdc451478a1ea8b1348f0ff240b20443a108bc9d0facd7b551daea2d2"}
Sep 14 12:58:48.449585 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.449562    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vftv8" event={"ID":"4cc3d786-1998-4dd8-b68d-739249a7ec18","Type":"ContainerStarted","Data":"26b63b95f3702ce180c5c1962a316c104aea5333a9aa4d6b19c84f5c33e6dec4"}
Sep 14 12:58:48.451450 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.451384    2581 generic.go:356] "Generic (PLEG): container finished" podID="19233420114c0f8b0b4e91a37e88ffae" containerID="a65269cbc1351262581047275579e38fa3a67204699247b1a935b741d81c4220" exitCode=0
Sep 14 12:58:48.451450 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.451406    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal" event={"ID":"19233420114c0f8b0b4e91a37e88ffae","Type":"ContainerDied","Data":"a65269cbc1351262581047275579e38fa3a67204699247b1a935b741d81c4220"}
Sep 14 12:58:48.452108 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.452038    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-njh47" podStartSLOduration=3.090343045 podStartE2EDuration="21.45202136s" totalImagesPullingTime="18.361678315s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-14 12:58:48.451786676 +0000 UTC m=+21.743120494" watchObservedRunningTime="2026-09-14 12:58:48.45202136 +0000 UTC m=+21.743355180"
Sep 14 12:58:48.465799 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.465756    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vftv8" podStartSLOduration=3.296811701 podStartE2EDuration="21.465741579s" totalImagesPullingTime="18.168929878s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:48.465352134 +0000 UTC m=+21.756685953" watchObservedRunningTime="2026-09-14 12:58:48.465741579 +0000 UTC m=+21.757075395"
Sep 14 12:58:48.484217 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.484167    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-sfhjz" podStartSLOduration=2.528302584 podStartE2EDuration="21.484151871s" totalImagesPullingTime="18.955849287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:48.483169583 +0000 UTC m=+21.774503400" watchObservedRunningTime="2026-09-14 12:58:48.484151871 +0000 UTC m=+21.775485688"
Sep 14 12:58:48.500600 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.500545    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9462f" podStartSLOduration=7.110683926 podStartE2EDuration="21.500516684s" totalImagesPullingTime="14.389832758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:48.50002013 +0000 UTC m=+21.791353947" watchObservedRunningTime="2026-09-14 12:58:48.500516684 +0000 UTC m=+21.791850501"
Sep 14 12:58:48.513893 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.513845    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-t797n" podStartSLOduration=3.4356296889999998 podStartE2EDuration="21.513828909s" totalImagesPullingTime="18.07819922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:48.513692114 +0000 UTC m=+21.805025932" watchObservedRunningTime="2026-09-14 12:58:48.513828909 +0000 UTC m=+21.805162727"
Sep 14 12:58:48.539218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.539183    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9462f"
Sep 14 12:58:48.539903 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.539880    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9462f"
Sep 14 12:58:48.612013 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.611987    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:48.745796 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.745719    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:48.841542 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.841352    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:48.951788 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:48.951694    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 14 12:58:49.239464 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:49.239436    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:49.239672 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:49.239607    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:49.239672 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:49.239655    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:49.239768 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:49.239749    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:49.299422 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:49.299305    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-14T12:58:48.95170962Z","UUID":"4c662380-0994-4ccc-979a-db4d2eebaf1d","Handler":null,"Name":"","Endpoint":""}
Sep 14 12:58:49.300943 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:49.300919    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 14 12:58:49.301067 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:49.300957    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 14 12:58:49.457803 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:49.457674    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76" event={"ID":"faf54388-f004-49f6-a2e0-3b7bf0d88ab6","Type":"ContainerStarted","Data":"e204eff8e474c0bef3751e2b61018828b13efbec7921593d1c636fd5e2454cda"}
Sep 14 12:58:49.461058 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:49.460924    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal" event={"ID":"19233420114c0f8b0b4e91a37e88ffae","Type":"ContainerStarted","Data":"795b09838bdf92afb090c3b66b14db62e93fbb2e6b821bc20c0d659e9f5ee3f7"}
Sep 14 12:58:49.477525 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:49.476764    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-35.ec2.internal" podStartSLOduration=21.476744905 podStartE2EDuration="21.476744905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:49.476411857 +0000 UTC m=+22.767745675" watchObservedRunningTime="2026-09-14 12:58:49.476744905 +0000 UTC m=+22.768078724"
Sep 14 12:58:50.240066 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:50.239979    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:50.240212 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:50.240129    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:50.465769 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:50.465724    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76" event={"ID":"faf54388-f004-49f6-a2e0-3b7bf0d88ab6","Type":"ContainerStarted","Data":"8ebe045bebef76bdd6b49706e93cb12cbd3905bf41834995ca936c5842649c88"}
Sep 14 12:58:50.469784 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:50.469756    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 12:58:50.470276 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:50.470248    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 12:58:50.470400 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:50.470248    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr" event={"ID":"9e5cd353-35cb-4693-a0e3-a49a546d8528","Type":"ContainerStarted","Data":"7835389c1e9ed4faf014920d2eea91e4606722f1bc7eafffac67d4f6b85ccc52"}
Sep 14 12:58:50.485211 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:50.485142    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjk76" podStartSLOduration=3.269875344 podStartE2EDuration="23.485120952s" totalImagesPullingTime="20.215245608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:50.484659499 +0000 UTC m=+23.775993319" watchObservedRunningTime="2026-09-14 12:58:50.485120952 +0000 UTC m=+23.776454770"
Sep 14 12:58:51.243565 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:51.243520    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:51.243770 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:51.243665    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:51.243770 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:51.243704    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:51.243888 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:51.243830    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:51.693724 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:51.693687    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9462f"
Sep 14 12:58:51.694132 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:51.693812    2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 12:58:51.694456 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:51.694433    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9462f"
Sep 14 12:58:51.766742 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:51.766698    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:52.239637 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:52.239595    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:52.239834 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:52.239747    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:53.240349 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:53.240253    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:53.240782 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:53.240390    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:53.240782 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:53.240439    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:53.240782 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:53.240551    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:53.479932 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:53.479904    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 12:58:53.480331 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:53.480309    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr" event={"ID":"9e5cd353-35cb-4693-a0e3-a49a546d8528","Type":"ContainerStarted","Data":"aec44efd4ff06b3f058d045b06155770c62af18fc95e0f9fc3e8a6c0235ded2d"}
Sep 14 12:58:53.480673 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:53.480650    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:53.480877 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:53.480827    2581 scope.go:117] "RemoveContainer" containerID="c9ed56d5ece1a0588a574de39eae1eed1ee0c9749f079f1d4591b3dcd7a13cb9"
Sep 14 12:58:53.482344 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:53.482325    2581 generic.go:356] "Generic (PLEG): container finished" podID="b59f7c6d-e692-4bb7-baca-3b4e6c932134" containerID="a5a3ffdce219fbeb46f777f774d4d75c79909a59c846a3e67912e79f09ae7ea9" exitCode=0
Sep 14 12:58:53.482428 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:53.482361    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdlhr" event={"ID":"b59f7c6d-e692-4bb7-baca-3b4e6c932134","Type":"ContainerDied","Data":"a5a3ffdce219fbeb46f777f774d4d75c79909a59c846a3e67912e79f09ae7ea9"}
Sep 14 12:58:53.497702 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:53.497674    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:53.792448 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:53.792350    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:53.895124 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:53.895088    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:54.239634 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.239600    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:54.239806 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:54.239732    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:54.488947 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.488918    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 12:58:54.489771 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.489313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr" event={"ID":"9e5cd353-35cb-4693-a0e3-a49a546d8528","Type":"ContainerStarted","Data":"6cc3f4e869655401736ba3d8275c661ef976a686ede874d4f348a89067b2e333"}
Sep 14 12:58:54.489771 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.489595    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:54.489771 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.489664    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:54.491774 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.491749    2581 generic.go:356] "Generic (PLEG): container finished" podID="b59f7c6d-e692-4bb7-baca-3b4e6c932134" containerID="b256f062e412637c2dd9e515b2b5d1a379e2bc860ec660037c0f2aab63d157d2" exitCode=0
Sep 14 12:58:54.491878 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.491794    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdlhr" event={"ID":"b59f7c6d-e692-4bb7-baca-3b4e6c932134","Type":"ContainerDied","Data":"b256f062e412637c2dd9e515b2b5d1a379e2bc860ec660037c0f2aab63d157d2"}
Sep 14 12:58:54.505469 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.505437    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:58:54.519102 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.519053    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr" podStartSLOduration=9.337527855 podStartE2EDuration="27.519037142s" totalImagesPullingTime="18.181509287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:54.517240972 +0000 UTC m=+27.808574788" watchObservedRunningTime="2026-09-14 12:58:54.519037142 +0000 UTC m=+27.810370958"
Sep 14 12:58:54.589342 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.589307    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-49fdc"]
Sep 14 12:58:54.589599 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.589460    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:54.589673 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:54.589607    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:54.590025 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.589988    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6p464"]
Sep 14 12:58:54.590138 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.590100    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:54.590222 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:54.590201    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:54.603934 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.603909    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dszvl"]
Sep 14 12:58:54.604050 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:54.604015    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:54.604122 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:54.604103    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:56.240212 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:56.240037    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:56.240669 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:56.240041    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:56.240669 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:56.240287    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:56.240669 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:56.240365    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:56.240669 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:56.240042    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:56.240669 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:56.240469    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:56.502388 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:56.502290    2581 generic.go:356] "Generic (PLEG): container finished" podID="b59f7c6d-e692-4bb7-baca-3b4e6c932134" containerID="7b9ac13e4a0e98e95b5cbee34dc6352c704dd790a22b0ef0157f89f560354aa5" exitCode=0
Sep 14 12:58:56.502388 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:56.502368    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdlhr" event={"ID":"b59f7c6d-e692-4bb7-baca-3b4e6c932134","Type":"ContainerDied","Data":"7b9ac13e4a0e98e95b5cbee34dc6352c704dd790a22b0ef0157f89f560354aa5"}
Sep 14 12:58:58.170446 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:58.170411    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:58.239591 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:58.239553    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:58.239591 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:58.239590    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:58.239853 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:58.239590    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:58.239853 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:58.239700    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-49fdc" podUID="f05d48a3-962c-42c0-aba5-1cd6441c74d9"
Sep 14 12:58:58.239853 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:58.239768    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6p464" podUID="f9624365-2f56-45f5-ba74-617be3896ac4"
Sep 14 12:58:58.240004 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:58.239855    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dszvl" podUID="db77a7f7-b9e7-47f8-a7d0-524847920a6b"
Sep 14 12:58:58.277902 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:58.277869    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:58.402052 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:58.402014    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:58.466192 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:58.466112    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:58.535044 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:58.535002    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:58.998692 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:58.998657    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-35.ec2.internal" event="NodeReady"
Sep 14 12:58:58.998877 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:58.998827    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 14 12:58:59.051879 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.051834    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xf7sp"]
Sep 14 12:58:59.101092 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.101052    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g2bqm"]
Sep 14 12:58:59.101273 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.101211    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xf7sp"
Sep 14 12:58:59.104547 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.104513    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 14 12:58:59.104689 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.104635    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5hqfk\""
Sep 14 12:58:59.104689 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.104644    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 14 12:58:59.104800 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.104758    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 14 12:58:59.114149 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.114129    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xf7sp"]
Sep 14 12:58:59.114285 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.114157    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g2bqm"]
Sep 14 12:58:59.114285 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.114263    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:58:59.117395 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.117373    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 14 12:58:59.117515 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.117413    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 14 12:58:59.117515 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.117443    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xjstd\""
Sep 14 12:58:59.118649 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.118625    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:59.218443 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.218361    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:59.252865 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.252826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-tmp-dir\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:58:59.253054 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.252881    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-config-volume\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:58:59.253054 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.252974    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:58:59.253176 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.253052    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9cec680f-1687-4f62-af5d-43a9c8f6f630-cert\") pod \"ingress-canary-xf7sp\" (UID: \"9cec680f-1687-4f62-af5d-43a9c8f6f630\") " pod="openshift-ingress-canary/ingress-canary-xf7sp"
Sep 14 12:58:59.253176 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.253086    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n8tt\" (UniqueName: \"kubernetes.io/projected/9cec680f-1687-4f62-af5d-43a9c8f6f630-kube-api-access-5n8tt\") pod \"ingress-canary-xf7sp\" (UID: \"9cec680f-1687-4f62-af5d-43a9c8f6f630\") " pod="openshift-ingress-canary/ingress-canary-xf7sp"
Sep 14 12:58:59.253176 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.253123    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sszr9\" (UniqueName: \"kubernetes.io/projected/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-kube-api-access-sszr9\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:58:59.272924 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.272881    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:59.354032 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.353997    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sszr9\" (UniqueName: \"kubernetes.io/projected/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-kube-api-access-sszr9\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:58:59.354212 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.354050    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-tmp-dir\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:58:59.354270 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.354218    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-config-volume\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:58:59.354323 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.354267    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:58:59.354323 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.354304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9cec680f-1687-4f62-af5d-43a9c8f6f630-cert\") pod \"ingress-canary-xf7sp\" (UID: \"9cec680f-1687-4f62-af5d-43a9c8f6f630\") " pod="openshift-ingress-canary/ingress-canary-xf7sp"
Sep 14 12:58:59.354477 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.354331    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n8tt\" (UniqueName: \"kubernetes.io/projected/9cec680f-1687-4f62-af5d-43a9c8f6f630-kube-api-access-5n8tt\") pod \"ingress-canary-xf7sp\" (UID: \"9cec680f-1687-4f62-af5d-43a9c8f6f630\") " pod="openshift-ingress-canary/ingress-canary-xf7sp"
Sep 14 12:58:59.354477 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:59.354393    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 12:58:59.354477 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:59.354452    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls podName:c32a3bdc-bd26-4891-843d-0808ffa7ec6d nodeName:}" failed. No retries permitted until 2026-09-14 12:58:59.854437643 +0000 UTC m=+33.145771466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls") pod "dns-default-g2bqm" (UID: "c32a3bdc-bd26-4891-843d-0808ffa7ec6d") : secret "dns-default-metrics-tls" not found
Sep 14 12:58:59.354657 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.354605    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-tmp-dir\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:58:59.355085 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.355059    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-config-volume\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:58:59.359001 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.358978    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9cec680f-1687-4f62-af5d-43a9c8f6f630-cert\") pod \"ingress-canary-xf7sp\" (UID: \"9cec680f-1687-4f62-af5d-43a9c8f6f630\") " pod="openshift-ingress-canary/ingress-canary-xf7sp"
Sep 14 12:58:59.363663 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.363643    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sszr9\" (UniqueName: \"kubernetes.io/projected/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-kube-api-access-sszr9\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:58:59.363946 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.363927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n8tt\" (UniqueName: \"kubernetes.io/projected/9cec680f-1687-4f62-af5d-43a9c8f6f630-kube-api-access-5n8tt\") pod \"ingress-canary-xf7sp\" (UID: \"9cec680f-1687-4f62-af5d-43a9c8f6f630\") " pod="openshift-ingress-canary/ingress-canary-xf7sp"
Sep 14 12:58:59.412844 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.412809    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xf7sp"
Sep 14 12:58:59.532493 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.532399    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:59.602874 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.602844    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:59.858339 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.858300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:58:59.858506 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.858346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs\") pod \"network-metrics-daemon-49fdc\" (UID: \"f05d48a3-962c-42c0-aba5-1cd6441c74d9\") " pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:58:59.858506 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.858376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:58:59.858506 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:59.858470    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:59.858506 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:59.858488    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 12:58:59.858695 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:59.858512    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:59.858695 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:59.858565    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls podName:c32a3bdc-bd26-4891-843d-0808ffa7ec6d nodeName:}" failed. No retries permitted until 2026-09-14 12:59:00.858551286 +0000 UTC m=+34.149885102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls") pod "dns-default-g2bqm" (UID: "c32a3bdc-bd26-4891-843d-0808ffa7ec6d") : secret "dns-default-metrics-tls" not found
Sep 14 12:58:59.858695 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:59.858580    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret podName:db77a7f7-b9e7-47f8-a7d0-524847920a6b nodeName:}" failed. No retries permitted until 2026-09-14 12:59:31.858574298 +0000 UTC m=+65.149908093 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret") pod "global-pull-secret-syncer-dszvl" (UID: "db77a7f7-b9e7-47f8-a7d0-524847920a6b") : object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:59.858695 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:59.858592    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs podName:f05d48a3-962c-42c0-aba5-1cd6441c74d9 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:31.858586525 +0000 UTC m=+65.149920319 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs") pod "network-metrics-daemon-49fdc" (UID: "f05d48a3-962c-42c0-aba5-1cd6441c74d9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:59.959699 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:58:59.959662    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mbnm\" (UniqueName: \"kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm\") pod \"network-check-target-6p464\" (UID: \"f9624365-2f56-45f5-ba74-617be3896ac4\") " pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:58:59.959878 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:59.959828    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 12:58:59.959878 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:59.959850    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 12:58:59.959878 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:59.959863    2581 projected.go:196] Error preparing data for projected volume kube-api-access-6mbnm for pod openshift-network-diagnostics/network-check-target-6p464: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:59.960055 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:58:59.959932    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm podName:f9624365-2f56-45f5-ba74-617be3896ac4 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:31.959914035 +0000 UTC m=+65.251247830 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6mbnm" (UniqueName: "kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm") pod "network-check-target-6p464" (UID: "f9624365-2f56-45f5-ba74-617be3896ac4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:59:00.033653 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:00.033619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:00.240155 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:00.239769    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:59:00.240155 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:00.239814    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:59:00.240155 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:00.239769    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:59:00.244785 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:00.244758    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 12:59:00.244785 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:00.244768    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 12:59:00.244785 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:00.244785    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 12:59:00.245094 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:00.245073    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tlghq\""
Sep 14 12:59:00.245210 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:00.245138    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 12:59:00.245299 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:00.245276    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9l5s5\""
Sep 14 12:59:00.653764 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:00.653589    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xf7sp"]
Sep 14 12:59:00.659740 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:00.659708    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cec680f_1687_4f62_af5d_43a9c8f6f630.slice/crio-d7b7c4f95c6fd9e1ca301e4ffd17ed7ce4d803d2fa75604d0c657631b4fd90bc WatchSource:0}: Error finding container d7b7c4f95c6fd9e1ca301e4ffd17ed7ce4d803d2fa75604d0c657631b4fd90bc: Status 404 returned error can't find the container with id d7b7c4f95c6fd9e1ca301e4ffd17ed7ce4d803d2fa75604d0c657631b4fd90bc
Sep 14 12:59:00.866382 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:00.866315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:59:00.866632 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:00.866434    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 12:59:00.866632 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:00.866487    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls podName:c32a3bdc-bd26-4891-843d-0808ffa7ec6d nodeName:}" failed. No retries permitted until 2026-09-14 12:59:02.86647284 +0000 UTC m=+36.157806648 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls") pod "dns-default-g2bqm" (UID: "c32a3bdc-bd26-4891-843d-0808ffa7ec6d") : secret "dns-default-metrics-tls" not found
Sep 14 12:59:01.517493 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:01.517456    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xf7sp" event={"ID":"9cec680f-1687-4f62-af5d-43a9c8f6f630","Type":"ContainerStarted","Data":"d7b7c4f95c6fd9e1ca301e4ffd17ed7ce4d803d2fa75604d0c657631b4fd90bc"}
Sep 14 12:59:01.521050 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:01.521009    2581 generic.go:356] "Generic (PLEG): container finished" podID="b59f7c6d-e692-4bb7-baca-3b4e6c932134" containerID="730b07f2dd77cf70185a23f00fcd2bb0d614ce791ee756defc71759f93af3afb" exitCode=0
Sep 14 12:59:01.521187 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:01.521063    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdlhr" event={"ID":"b59f7c6d-e692-4bb7-baca-3b4e6c932134","Type":"ContainerDied","Data":"730b07f2dd77cf70185a23f00fcd2bb0d614ce791ee756defc71759f93af3afb"}
Sep 14 12:59:02.527493 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:02.527454    2581 generic.go:356] "Generic (PLEG): container finished" podID="b59f7c6d-e692-4bb7-baca-3b4e6c932134" containerID="17a23b6606080f5b9a481bc5821fe92af0c0c241e6bdd3f252d7a89671fdf236" exitCode=0
Sep 14 12:59:02.528041 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:02.527514    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdlhr" event={"ID":"b59f7c6d-e692-4bb7-baca-3b4e6c932134","Type":"ContainerDied","Data":"17a23b6606080f5b9a481bc5821fe92af0c0c241e6bdd3f252d7a89671fdf236"}
Sep 14 12:59:02.558450 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:02.558419    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:02.808767 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:02.808683    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:02.882806 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:02.882769    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:59:02.882971 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:02.882899    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 12:59:02.883025 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:02.882979    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls podName:c32a3bdc-bd26-4891-843d-0808ffa7ec6d nodeName:}" failed. No retries permitted until 2026-09-14 12:59:06.882964213 +0000 UTC m=+40.174298012 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls") pod "dns-default-g2bqm" (UID: "c32a3bdc-bd26-4891-843d-0808ffa7ec6d") : secret "dns-default-metrics-tls" not found
Sep 14 12:59:03.533917 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:03.533890    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdlhr" event={"ID":"b59f7c6d-e692-4bb7-baca-3b4e6c932134","Type":"ContainerStarted","Data":"7e52a46f3221db8e93fb55d40df1ee4c6afedd2a767a8267d80ee66603fe1879"}
Sep 14 12:59:03.556818 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:03.556777    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zdlhr" podStartSLOduration=8.21889823 podStartE2EDuration="36.556759346s" totalImagesPullingTime="28.337861116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:03.554698317 +0000 UTC m=+36.846032147" watchObservedRunningTime="2026-09-14 12:59:03.556759346 +0000 UTC m=+36.848093163"
Sep 14 12:59:03.679729 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:03.679691    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:03.739219 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:03.739187    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:04.537631 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:04.537546    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xf7sp" event={"ID":"9cec680f-1687-4f62-af5d-43a9c8f6f630","Type":"ContainerStarted","Data":"9fdfd27e5bf1ad7f77c079a37e1f0aa7fa7fe622e1b71369fccb4f3c61f4a9ac"}
Sep 14 12:59:04.554063 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:04.554010    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xf7sp" podStartSLOduration=2.718353381 podStartE2EDuration="5.553988645s" totalImagesPullingTime="2.835635264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:04.553654612 +0000 UTC m=+37.844988428" watchObservedRunningTime="2026-09-14 12:59:04.553988645 +0000 UTC m=+37.845322461"
Sep 14 12:59:05.291185 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:05.291150    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:05.347099 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:05.347067    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:06.290677 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:06.290643    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:06.424745 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:06.424714    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:06.911868 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:06.911827    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:59:06.912070 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:06.911982    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 12:59:06.912070 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:06.912047    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls podName:c32a3bdc-bd26-4891-843d-0808ffa7ec6d nodeName:}" failed. No retries permitted until 2026-09-14 12:59:14.912030431 +0000 UTC m=+48.203364226 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls") pod "dns-default-g2bqm" (UID: "c32a3bdc-bd26-4891-843d-0808ffa7ec6d") : secret "dns-default-metrics-tls" not found
Sep 14 12:59:07.359272 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.359228    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:07.368774 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.368742    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-r8tc5"]
Sep 14 12:59:07.391421 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.391394    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-r8tc5"]
Sep 14 12:59:07.391609 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.391542    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-r8tc5"
Sep 14 12:59:07.394623 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.394601    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 14 12:59:07.395944 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.395927    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 14 12:59:07.396022 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.395927    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kmbkk\""
Sep 14 12:59:07.442768 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.442724    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:07.512770 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.512737    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:07.516708 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.516687    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7lhk\" (UniqueName: \"kubernetes.io/projected/f9e5b8f0-56f7-4bf9-bfe2-e420ec105e4c-kube-api-access-n7lhk\") pod \"migrator-8b4d697c5-r8tc5\" (UID: \"f9e5b8f0-56f7-4bf9-bfe2-e420ec105e4c\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-r8tc5"
Sep 14 12:59:07.573408 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.573375    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:07.618133 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.618044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7lhk\" (UniqueName: \"kubernetes.io/projected/f9e5b8f0-56f7-4bf9-bfe2-e420ec105e4c-kube-api-access-n7lhk\") pod \"migrator-8b4d697c5-r8tc5\" (UID: \"f9e5b8f0-56f7-4bf9-bfe2-e420ec105e4c\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-r8tc5"
Sep 14 12:59:07.631805 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.631772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7lhk\" (UniqueName: \"kubernetes.io/projected/f9e5b8f0-56f7-4bf9-bfe2-e420ec105e4c-kube-api-access-n7lhk\") pod \"migrator-8b4d697c5-r8tc5\" (UID: \"f9e5b8f0-56f7-4bf9-bfe2-e420ec105e4c\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-r8tc5"
Sep 14 12:59:07.644357 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.644323    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bnpj9"]
Sep 14 12:59:07.659112 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.659081    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:07.660964 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.660937    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bnpj9"]
Sep 14 12:59:07.661082 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.661067    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:07.664037 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.664012    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 14 12:59:07.664131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.664036    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-26vf9\""
Sep 14 12:59:07.664131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.664013    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 14 12:59:07.664131 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.664013    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 14 12:59:07.665511 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.665493    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 14 12:59:07.701871 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.701833    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-r8tc5"
Sep 14 12:59:07.819984 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.819943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c20fe060-6c64-4048-ac0c-bf214312d5ad-crio-socket\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:07.820155 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.820000    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:07.820155 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.820054    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgw96\" (UniqueName: \"kubernetes.io/projected/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-api-access-sgw96\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:07.820155 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.820076    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:07.820155 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.820112    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c20fe060-6c64-4048-ac0c-bf214312d5ad-data-volume\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:07.921478 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.921384    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c20fe060-6c64-4048-ac0c-bf214312d5ad-data-volume\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:07.921478 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.921451    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c20fe060-6c64-4048-ac0c-bf214312d5ad-crio-socket\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:07.921478 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.921478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:07.921756 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.921505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgw96\" (UniqueName: \"kubernetes.io/projected/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-api-access-sgw96\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:07.921756 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.921522    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:07.921756 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:07.921634    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 12:59:07.921756 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:07.921665    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:07.921756 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:07.921717    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm podName:c20fe060-6c64-4048-ac0c-bf214312d5ad nodeName:}" failed. No retries permitted until 2026-09-14 12:59:08.421699808 +0000 UTC m=+41.713033618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bnpj9" (UID: "c20fe060-6c64-4048-ac0c-bf214312d5ad") : configmap "kube-rbac-proxy" not found
Sep 14 12:59:07.921756 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:07.921732    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls podName:c20fe060-6c64-4048-ac0c-bf214312d5ad nodeName:}" failed. No retries permitted until 2026-09-14 12:59:08.421725921 +0000 UTC m=+41.713059716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bnpj9" (UID: "c20fe060-6c64-4048-ac0c-bf214312d5ad") : secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:07.921995 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.921756    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c20fe060-6c64-4048-ac0c-bf214312d5ad-crio-socket\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:07.921995 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.921832    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c20fe060-6c64-4048-ac0c-bf214312d5ad-data-volume\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:07.931018 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.930997    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgw96\" (UniqueName: \"kubernetes.io/projected/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-api-access-sgw96\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:07.953187 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.953155    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:07.993092 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:07.993056    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-r8tc5"]
Sep 14 12:59:07.996110 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:07.996087    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e5b8f0_56f7_4bf9_bfe2_e420ec105e4c.slice/crio-633bcf094b85d0af6d6852b5268c68b0dd503a26f43c9f9da5c2ea87373b4e12 WatchSource:0}: Error finding container 633bcf094b85d0af6d6852b5268c68b0dd503a26f43c9f9da5c2ea87373b4e12: Status 404 returned error can't find the container with id 633bcf094b85d0af6d6852b5268c68b0dd503a26f43c9f9da5c2ea87373b4e12
Sep 14 12:59:08.054388 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:08.054356    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:08.305515 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:08.305419    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:08.405041 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:08.405008    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:08.425225 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:08.425191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:08.425334 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:08.425243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:08.425334 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:08.425308    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 12:59:08.425399 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:08.425357    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:08.425432 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:08.425409    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls podName:c20fe060-6c64-4048-ac0c-bf214312d5ad nodeName:}" failed. No retries permitted until 2026-09-14 12:59:09.42539525 +0000 UTC m=+42.716729044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bnpj9" (UID: "c20fe060-6c64-4048-ac0c-bf214312d5ad") : secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:08.425432 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:08.425429    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm podName:c20fe060-6c64-4048-ac0c-bf214312d5ad nodeName:}" failed. No retries permitted until 2026-09-14 12:59:09.425419712 +0000 UTC m=+42.716753507 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bnpj9" (UID: "c20fe060-6c64-4048-ac0c-bf214312d5ad") : configmap "kube-rbac-proxy" not found
Sep 14 12:59:08.548765 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:08.548727    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-r8tc5" event={"ID":"f9e5b8f0-56f7-4bf9-bfe2-e420ec105e4c","Type":"ContainerStarted","Data":"633bcf094b85d0af6d6852b5268c68b0dd503a26f43c9f9da5c2ea87373b4e12"}
Sep 14 12:59:09.435347 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.435308    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:09.435347 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.435364    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:09.435848 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:09.435450    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 12:59:09.435848 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:09.435464    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:09.435848 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:09.435518    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm podName:c20fe060-6c64-4048-ac0c-bf214312d5ad nodeName:}" failed. No retries permitted until 2026-09-14 12:59:11.435503588 +0000 UTC m=+44.726837383 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bnpj9" (UID: "c20fe060-6c64-4048-ac0c-bf214312d5ad") : configmap "kube-rbac-proxy" not found
Sep 14 12:59:09.435848 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:09.435570    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls podName:c20fe060-6c64-4048-ac0c-bf214312d5ad nodeName:}" failed. No retries permitted until 2026-09-14 12:59:11.435550548 +0000 UTC m=+44.726884351 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bnpj9" (UID: "c20fe060-6c64-4048-ac0c-bf214312d5ad") : secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:09.828160 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.828130    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-jqm7l"]
Sep 14 12:59:09.848490 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.848461    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-jqm7l"]
Sep 14 12:59:09.848650 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.848578    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-jqm7l"
Sep 14 12:59:09.851702 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.851680    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 14 12:59:09.851804 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.851719    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 14 12:59:09.852680 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.852665    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 14 12:59:09.852719 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.852683    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-m2szw\""
Sep 14 12:59:09.852932 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.852918    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 14 12:59:09.861085 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.861061    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:09.940705 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.940674    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/df979d45-ad7a-4797-8517-e27647899499-signing-cabundle\") pod \"service-ca-568869fdc5-jqm7l\" (UID: \"df979d45-ad7a-4797-8517-e27647899499\") " pod="openshift-service-ca/service-ca-568869fdc5-jqm7l"
Sep 14 12:59:09.940917 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.940729    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/df979d45-ad7a-4797-8517-e27647899499-signing-key\") pod \"service-ca-568869fdc5-jqm7l\" (UID: \"df979d45-ad7a-4797-8517-e27647899499\") " pod="openshift-service-ca/service-ca-568869fdc5-jqm7l"
Sep 14 12:59:09.940917 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.940823    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5r7g\" (UniqueName: \"kubernetes.io/projected/df979d45-ad7a-4797-8517-e27647899499-kube-api-access-b5r7g\") pod \"service-ca-568869fdc5-jqm7l\" (UID: \"df979d45-ad7a-4797-8517-e27647899499\") " pod="openshift-service-ca/service-ca-568869fdc5-jqm7l"
Sep 14 12:59:09.961754 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:09.961685    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:10.041993 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.041948    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/df979d45-ad7a-4797-8517-e27647899499-signing-cabundle\") pod \"service-ca-568869fdc5-jqm7l\" (UID: \"df979d45-ad7a-4797-8517-e27647899499\") " pod="openshift-service-ca/service-ca-568869fdc5-jqm7l"
Sep 14 12:59:10.042148 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.042017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/df979d45-ad7a-4797-8517-e27647899499-signing-key\") pod \"service-ca-568869fdc5-jqm7l\" (UID: \"df979d45-ad7a-4797-8517-e27647899499\") " pod="openshift-service-ca/service-ca-568869fdc5-jqm7l"
Sep 14 12:59:10.042148 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.042112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5r7g\" (UniqueName: \"kubernetes.io/projected/df979d45-ad7a-4797-8517-e27647899499-kube-api-access-b5r7g\") pod \"service-ca-568869fdc5-jqm7l\" (UID: \"df979d45-ad7a-4797-8517-e27647899499\") " pod="openshift-service-ca/service-ca-568869fdc5-jqm7l"
Sep 14 12:59:10.042706 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.042688    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/df979d45-ad7a-4797-8517-e27647899499-signing-cabundle\") pod \"service-ca-568869fdc5-jqm7l\" (UID: \"df979d45-ad7a-4797-8517-e27647899499\") " pod="openshift-service-ca/service-ca-568869fdc5-jqm7l"
Sep 14 12:59:10.044525 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.044491    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/df979d45-ad7a-4797-8517-e27647899499-signing-key\") pod \"service-ca-568869fdc5-jqm7l\" (UID: \"df979d45-ad7a-4797-8517-e27647899499\") " pod="openshift-service-ca/service-ca-568869fdc5-jqm7l"
Sep 14 12:59:10.050471 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.050445    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5r7g\" (UniqueName: \"kubernetes.io/projected/df979d45-ad7a-4797-8517-e27647899499-kube-api-access-b5r7g\") pod \"service-ca-568869fdc5-jqm7l\" (UID: \"df979d45-ad7a-4797-8517-e27647899499\") " pod="openshift-service-ca/service-ca-568869fdc5-jqm7l"
Sep 14 12:59:10.164000 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.163955    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-jqm7l"
Sep 14 12:59:10.448312 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.448283    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-jqm7l"]
Sep 14 12:59:10.450928 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:10.450903    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf979d45_ad7a_4797_8517_e27647899499.slice/crio-78c8272d7e5987fb6d2779e3ddd263ff5d34e50df1e4dccc65490ca35d506e31 WatchSource:0}: Error finding container 78c8272d7e5987fb6d2779e3ddd263ff5d34e50df1e4dccc65490ca35d506e31: Status 404 returned error can't find the container with id 78c8272d7e5987fb6d2779e3ddd263ff5d34e50df1e4dccc65490ca35d506e31
Sep 14 12:59:10.556318 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.556224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-r8tc5" event={"ID":"f9e5b8f0-56f7-4bf9-bfe2-e420ec105e4c","Type":"ContainerStarted","Data":"b105db355947fda330fda835c59b5f8911eb51da4bfd62957228ffebaca4947a"}
Sep 14 12:59:10.556318 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.556289    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-r8tc5" event={"ID":"f9e5b8f0-56f7-4bf9-bfe2-e420ec105e4c","Type":"ContainerStarted","Data":"4dc46a7a9c93e3fc52ff25d1683061b6b5620c106407de87bf07a57db026ef50"}
Sep 14 12:59:10.557388 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.557364    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-jqm7l" event={"ID":"df979d45-ad7a-4797-8517-e27647899499","Type":"ContainerStarted","Data":"78c8272d7e5987fb6d2779e3ddd263ff5d34e50df1e4dccc65490ca35d506e31"}
Sep 14 12:59:10.573126 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.573078    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-r8tc5" podStartSLOduration=1.891639372 podStartE2EDuration="3.573060646s" totalImagesPullingTime="1.681421274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:10.572898717 +0000 UTC m=+43.864232536" watchObservedRunningTime="2026-09-14 12:59:10.573060646 +0000 UTC m=+43.864394465"
Sep 14 12:59:10.596965 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.596938    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vftv8_4cc3d786-1998-4dd8-b68d-739249a7ec18/dns-node-resolver/0.log"
Sep 14 12:59:10.652717 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.652687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:10.753863 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:10.753828    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:11.426203 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:11.426165    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:11.452233 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:11.452187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:11.452646 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:11.452331    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:11.452646 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:11.452338    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:11.452646 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:11.452369    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 12:59:11.452646 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:11.452438    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls podName:c20fe060-6c64-4048-ac0c-bf214312d5ad nodeName:}" failed. No retries permitted until 2026-09-14 12:59:15.4524212 +0000 UTC m=+48.743754994 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bnpj9" (UID: "c20fe060-6c64-4048-ac0c-bf214312d5ad") : secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:11.452646 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:11.452466    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm podName:c20fe060-6c64-4048-ac0c-bf214312d5ad nodeName:}" failed. No retries permitted until 2026-09-14 12:59:15.452445415 +0000 UTC m=+48.743779216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bnpj9" (UID: "c20fe060-6c64-4048-ac0c-bf214312d5ad") : configmap "kube-rbac-proxy" not found
Sep 14 12:59:11.596886 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:11.596829    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t797n_8e6b0822-d820-47f4-a652-f56f2f95707e/node-ca/0.log"
Sep 14 12:59:12.396798 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:12.396768    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xf7sp_9cec680f-1687-4f62-af5d-43a9c8f6f630/serve-healthcheck-canary/0.log"
Sep 14 12:59:12.597628 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:12.597600    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-r8tc5_f9e5b8f0-56f7-4bf9-bfe2-e420ec105e4c/migrator/0.log"
Sep 14 12:59:12.796059 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:12.795984    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-r8tc5_f9e5b8f0-56f7-4bf9-bfe2-e420ec105e4c/graceful-termination/0.log"
Sep 14 12:59:13.569905 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:13.569862    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-jqm7l" event={"ID":"df979d45-ad7a-4797-8517-e27647899499","Type":"ContainerStarted","Data":"0a2fe32c99504fbeec3bef3a67ea084332f542e0016e407ebabe5fa17bf88a1c"}
Sep 14 12:59:13.594345 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:13.594287    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-jqm7l" podStartSLOduration=2.236338132 podStartE2EDuration="4.594269801s" totalImagesPullingTime="2.357931669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:13.593139325 +0000 UTC m=+46.884473143" watchObservedRunningTime="2026-09-14 12:59:13.594269801 +0000 UTC m=+46.885603669"
Sep 14 12:59:14.587720 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:14.587685    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:14.690045 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:14.690012    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:14.912905 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:14.912856    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:14.979442 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:14.979405    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:59:14.979626 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:14.979584    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 12:59:14.979674 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:14.979653    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls podName:c32a3bdc-bd26-4891-843d-0808ffa7ec6d nodeName:}" failed. No retries permitted until 2026-09-14 12:59:30.979637703 +0000 UTC m=+64.270971533 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls") pod "dns-default-g2bqm" (UID: "c32a3bdc-bd26-4891-843d-0808ffa7ec6d") : secret "dns-default-metrics-tls" not found
Sep 14 12:59:15.071100 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:15.071062    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:15.483220 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:15.483159    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:15.483427 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:15.483236    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:15.483427 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:15.483305    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 12:59:15.483427 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:15.483389    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:15.483427 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:15.483394    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm podName:c20fe060-6c64-4048-ac0c-bf214312d5ad nodeName:}" failed. No retries permitted until 2026-09-14 12:59:23.483371738 +0000 UTC m=+56.774705546 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bnpj9" (UID: "c20fe060-6c64-4048-ac0c-bf214312d5ad") : configmap "kube-rbac-proxy" not found
Sep 14 12:59:15.483619 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:15.483441    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls podName:c20fe060-6c64-4048-ac0c-bf214312d5ad nodeName:}" failed. No retries permitted until 2026-09-14 12:59:23.483430303 +0000 UTC m=+56.774764102 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bnpj9" (UID: "c20fe060-6c64-4048-ac0c-bf214312d5ad") : secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:16.068940 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:16.068903    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:16.262176 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:16.262143    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:16.663036 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:16.663006    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:16.862211 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:16.862167    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:17.069516 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:17.069401    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:17.556317 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:17.556277    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:17.882989 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:17.882955    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:18.070848 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:18.070814    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:18.258700 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:18.258620    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:18.616694 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:18.616667    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:18.717136 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:18.717104    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:18.862686 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:18.862654    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:19.269655 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:19.269621    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:20.063853 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:20.063815    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:20.466837 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:20.466805    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:20.569382 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:20.569349    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:20.667276 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:20.667239    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:20.767694 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:20.767615    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:20.867665 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:20.867629    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:21.068119 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:21.068040    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:22.071128 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:22.071094    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:22.266288 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:22.266255    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:22.661663 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:22.661612    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:22.861796 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:22.861754    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:23.262719 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:23.262680    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:23.546665 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:23.546573    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:23.546665 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:23.546626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:23.546918 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:23.546726    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:23.546918 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:23.546789    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls podName:c20fe060-6c64-4048-ac0c-bf214312d5ad nodeName:}" failed. No retries permitted until 2026-09-14 12:59:39.546776304 +0000 UTC m=+72.838110112 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bnpj9" (UID: "c20fe060-6c64-4048-ac0c-bf214312d5ad") : secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:23.547110 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:23.547086    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c20fe060-6c64-4048-ac0c-bf214312d5ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:23.723184 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:23.723150    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:23.819588 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:23.819490    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:23.917763 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:23.917726    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:24.017682 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:24.017649    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:24.363798 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:24.363766    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:24.464091 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:24.464060    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:24.564992 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:24.564959    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:24.618670 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:24.618596    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:24.677480 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:24.677453    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:24.779233 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:24.779209    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:24.878416 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:24.878339    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:24.978823 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:24.978776    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:25.082680 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:25.082649    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:25.183477 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:25.183401    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:25.280825 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:25.280798    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:25.380416 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:25.380386    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:25.565972 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:25.565877    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:25.734323 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:25.734290    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:26.513453 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:26.513421    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d8mhr"
Sep 14 12:59:26.589663 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:26.589630    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:26.690596 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:26.690562    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:31.002091 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:31.002035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:59:31.004303 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:31.004283    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c32a3bdc-bd26-4891-843d-0808ffa7ec6d-metrics-tls\") pod \"dns-default-g2bqm\" (UID: \"c32a3bdc-bd26-4891-843d-0808ffa7ec6d\") " pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:59:31.227367 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:31.227330    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xjstd\""
Sep 14 12:59:31.234780 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:31.234757    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:59:31.535745 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:31.535675    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g2bqm"]
Sep 14 12:59:31.538589 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:31.538561    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc32a3bdc_bd26_4891_843d_0808ffa7ec6d.slice/crio-1a1f29268be57d78413606e5bed5e9bd15c29c5e6fbd7e9062e4858b6bf6fe24 WatchSource:0}: Error finding container 1a1f29268be57d78413606e5bed5e9bd15c29c5e6fbd7e9062e4858b6bf6fe24: Status 404 returned error can't find the container with id 1a1f29268be57d78413606e5bed5e9bd15c29c5e6fbd7e9062e4858b6bf6fe24
Sep 14 12:59:31.617922 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:31.617890    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2bqm" event={"ID":"c32a3bdc-bd26-4891-843d-0808ffa7ec6d","Type":"ContainerStarted","Data":"1a1f29268be57d78413606e5bed5e9bd15c29c5e6fbd7e9062e4858b6bf6fe24"}
Sep 14 12:59:31.907580 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:31.907524    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:59:31.907723 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:31.907589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs\") pod \"network-metrics-daemon-49fdc\" (UID: \"f05d48a3-962c-42c0-aba5-1cd6441c74d9\") " pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:59:31.910055 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:31.910036    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 12:59:31.910755 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:31.910740    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 12:59:31.920711 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:31.920694    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/db77a7f7-b9e7-47f8-a7d0-524847920a6b-original-pull-secret\") pod \"global-pull-secret-syncer-dszvl\" (UID: \"db77a7f7-b9e7-47f8-a7d0-524847920a6b\") " pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:59:31.920784 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:31.920718    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f05d48a3-962c-42c0-aba5-1cd6441c74d9-metrics-certs\") pod \"network-metrics-daemon-49fdc\" (UID: \"f05d48a3-962c-42c0-aba5-1cd6441c74d9\") " pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:59:32.008238 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.008201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mbnm\" (UniqueName: \"kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm\") pod \"network-check-target-6p464\" (UID: \"f9624365-2f56-45f5-ba74-617be3896ac4\") " pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:59:32.010723 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.010700    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 12:59:32.021031 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.021006    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 12:59:32.031159 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.031128    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mbnm\" (UniqueName: \"kubernetes.io/projected/f9624365-2f56-45f5-ba74-617be3896ac4-kube-api-access-6mbnm\") pod \"network-check-target-6p464\" (UID: \"f9624365-2f56-45f5-ba74-617be3896ac4\") " pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:59:32.055497 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.055468    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tlghq\""
Sep 14 12:59:32.063596 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.063577    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:59:32.063868 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.063852    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9l5s5\""
Sep 14 12:59:32.067233 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.067216    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dszvl"
Sep 14 12:59:32.071970 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.071948    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-49fdc"
Sep 14 12:59:32.337057 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.337026    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-swpmp"]
Sep 14 12:59:32.339098 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.339076    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-swpmp"
Sep 14 12:59:32.342796 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.342766    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 14 12:59:32.342975 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.342957    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9k7k4\""
Sep 14 12:59:32.343099 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.343072    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 14 12:59:32.358864 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.358806    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-swpmp"]
Sep 14 12:59:32.359377 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.359351    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:32.380722 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.380685    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6p464"]
Sep 14 12:59:32.381183 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.381163    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dszvl"]
Sep 14 12:59:32.382281 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:32.381596    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9624365_2f56_45f5_ba74_617be3896ac4.slice/crio-2532a244b3142aa83ef967dc6beb8191703cf169bb647eedabb8ef7698bacfaa WatchSource:0}: Error finding container 2532a244b3142aa83ef967dc6beb8191703cf169bb647eedabb8ef7698bacfaa: Status 404 returned error can't find the container with id 2532a244b3142aa83ef967dc6beb8191703cf169bb647eedabb8ef7698bacfaa
Sep 14 12:59:32.382281 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.382065    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-49fdc"]
Sep 14 12:59:32.383817 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:32.383769    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf05d48a3_962c_42c0_aba5_1cd6441c74d9.slice/crio-ca43922dd05b1ba2394a1813198c7ccb51e2a58962a098c888be79742d4412f8 WatchSource:0}: Error finding container ca43922dd05b1ba2394a1813198c7ccb51e2a58962a098c888be79742d4412f8: Status 404 returned error can't find the container with id ca43922dd05b1ba2394a1813198c7ccb51e2a58962a098c888be79742d4412f8
Sep 14 12:59:32.385152 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:32.385131    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb77a7f7_b9e7_47f8_a7d0_524847920a6b.slice/crio-0d6382495e3e471bcd6ad949dd1c76babbc45fb582e58e620908f76d4dc12f1b WatchSource:0}: Error finding container 0d6382495e3e471bcd6ad949dd1c76babbc45fb582e58e620908f76d4dc12f1b: Status 404 returned error can't find the container with id 0d6382495e3e471bcd6ad949dd1c76babbc45fb582e58e620908f76d4dc12f1b
Sep 14 12:59:32.412185 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.412127    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2bwt\" (UniqueName: \"kubernetes.io/projected/229dfed7-2ade-4f61-8c09-333b42988051-kube-api-access-k2bwt\") pod \"downloads-76f99584f4-swpmp\" (UID: \"229dfed7-2ade-4f61-8c09-333b42988051\") " pod="openshift-console/downloads-76f99584f4-swpmp"
Sep 14 12:59:32.424976 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.424953    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"]
Sep 14 12:59:32.427620 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.427597    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.430062 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.430042    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8dx6q\""
Sep 14 12:59:32.430670 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.430654    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 14 12:59:32.430892 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.430874    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 14 12:59:32.431020 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.430921    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 14 12:59:32.437851 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.437828    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 14 12:59:32.448196 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.448172    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"]
Sep 14 12:59:32.481844 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.481809    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:32.513498 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.513467    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0970c76-9964-4645-b9a9-92aa45a58fc0-registry-tls\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.513498 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.513501    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c0970c76-9964-4645-b9a9-92aa45a58fc0-ca-trust-extracted\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.513760 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.513539    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c0970c76-9964-4645-b9a9-92aa45a58fc0-trusted-ca\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.513760 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.513590    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2bwt\" (UniqueName: \"kubernetes.io/projected/229dfed7-2ade-4f61-8c09-333b42988051-kube-api-access-k2bwt\") pod \"downloads-76f99584f4-swpmp\" (UID: \"229dfed7-2ade-4f61-8c09-333b42988051\") " pod="openshift-console/downloads-76f99584f4-swpmp"
Sep 14 12:59:32.513760 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.513609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0970c76-9964-4645-b9a9-92aa45a58fc0-bound-sa-token\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.513760 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.513626    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6m24\" (UniqueName: \"kubernetes.io/projected/c0970c76-9964-4645-b9a9-92aa45a58fc0-kube-api-access-z6m24\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.513760 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.513660    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c0970c76-9964-4645-b9a9-92aa45a58fc0-registry-certificates\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.513760 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.513687    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c0970c76-9964-4645-b9a9-92aa45a58fc0-image-registry-private-configuration\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.513760 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.513704    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c0970c76-9964-4645-b9a9-92aa45a58fc0-installation-pull-secrets\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.523990 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.523968    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2bwt\" (UniqueName: \"kubernetes.io/projected/229dfed7-2ade-4f61-8c09-333b42988051-kube-api-access-k2bwt\") pod \"downloads-76f99584f4-swpmp\" (UID: \"229dfed7-2ade-4f61-8c09-333b42988051\") " pod="openshift-console/downloads-76f99584f4-swpmp"
Sep 14 12:59:32.568079 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.568050    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:32.614086 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.614006    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c0970c76-9964-4645-b9a9-92aa45a58fc0-image-registry-private-configuration\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.614086 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.614052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c0970c76-9964-4645-b9a9-92aa45a58fc0-installation-pull-secrets\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.614315 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.614115    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0970c76-9964-4645-b9a9-92aa45a58fc0-registry-tls\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.614315 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.614139    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c0970c76-9964-4645-b9a9-92aa45a58fc0-ca-trust-extracted\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.614315 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.614166    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c0970c76-9964-4645-b9a9-92aa45a58fc0-trusted-ca\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.614315 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.614205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0970c76-9964-4645-b9a9-92aa45a58fc0-bound-sa-token\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.614315 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.614228    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6m24\" (UniqueName: \"kubernetes.io/projected/c0970c76-9964-4645-b9a9-92aa45a58fc0-kube-api-access-z6m24\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.614315 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.614262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c0970c76-9964-4645-b9a9-92aa45a58fc0-registry-certificates\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.615155 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.615125    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c0970c76-9964-4645-b9a9-92aa45a58fc0-ca-trust-extracted\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.615680 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.615657    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c0970c76-9964-4645-b9a9-92aa45a58fc0-registry-certificates\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.617210 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.617180    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c0970c76-9964-4645-b9a9-92aa45a58fc0-trusted-ca\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.618240 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.618214    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c0970c76-9964-4645-b9a9-92aa45a58fc0-registry-tls\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.620420 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.620394    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c0970c76-9964-4645-b9a9-92aa45a58fc0-installation-pull-secrets\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.620941 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.620918    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c0970c76-9964-4645-b9a9-92aa45a58fc0-image-registry-private-configuration\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.623599 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.623567    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6p464" event={"ID":"f9624365-2f56-45f5-ba74-617be3896ac4","Type":"ContainerStarted","Data":"2532a244b3142aa83ef967dc6beb8191703cf169bb647eedabb8ef7698bacfaa"}
Sep 14 12:59:32.624960 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.624926    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dszvl" event={"ID":"db77a7f7-b9e7-47f8-a7d0-524847920a6b","Type":"ContainerStarted","Data":"0d6382495e3e471bcd6ad949dd1c76babbc45fb582e58e620908f76d4dc12f1b"}
Sep 14 12:59:32.626485 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.626457    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-49fdc" event={"ID":"f05d48a3-962c-42c0-aba5-1cd6441c74d9","Type":"ContainerStarted","Data":"ca43922dd05b1ba2394a1813198c7ccb51e2a58962a098c888be79742d4412f8"}
Sep 14 12:59:32.626778 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.626755    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6m24\" (UniqueName: \"kubernetes.io/projected/c0970c76-9964-4645-b9a9-92aa45a58fc0-kube-api-access-z6m24\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.626872 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.626839    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c0970c76-9964-4645-b9a9-92aa45a58fc0-bound-sa-token\") pod \"image-registry-7fcb6fd65f-jkfsm\" (UID: \"c0970c76-9964-4645-b9a9-92aa45a58fc0\") " pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.651692 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.651657    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-swpmp"
Sep 14 12:59:32.667934 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.667855    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:32.738259 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.738224    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:32.767918 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.767884    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:32.929808 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:32.929685    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-swpmp"]
Sep 14 12:59:33.441450 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:33.441413    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod229dfed7_2ade_4f61_8c09_333b42988051.slice/crio-8b5f3a220af114efce95d6df1656c310ae858a05484dbf9a6b4a5e2e78131a2b WatchSource:0}: Error finding container 8b5f3a220af114efce95d6df1656c310ae858a05484dbf9a6b4a5e2e78131a2b: Status 404 returned error can't find the container with id 8b5f3a220af114efce95d6df1656c310ae858a05484dbf9a6b4a5e2e78131a2b
Sep 14 12:59:33.605702 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:33.605657    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:33.633849 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:33.633815    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2bqm" event={"ID":"c32a3bdc-bd26-4891-843d-0808ffa7ec6d","Type":"ContainerStarted","Data":"df7321d9fc8f1d7c0ba66e92df8e800ccbc9f84bef05d2f3d17c5ea0d36d7c39"}
Sep 14 12:59:33.641218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:33.641182    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-49fdc" event={"ID":"f05d48a3-962c-42c0-aba5-1cd6441c74d9","Type":"ContainerStarted","Data":"35bb2d418163827a1b5eb19078410dff660ef4827f2b74be0006d97fcc4b8a30"}
Sep 14 12:59:33.647031 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:33.646963    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-swpmp" event={"ID":"229dfed7-2ade-4f61-8c09-333b42988051","Type":"ContainerStarted","Data":"8b5f3a220af114efce95d6df1656c310ae858a05484dbf9a6b4a5e2e78131a2b"}
Sep 14 12:59:33.705547 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:33.704564    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:33.759752 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:33.759358    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp"]
Sep 14 12:59:33.762487 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:33.762460    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp"
Sep 14 12:59:33.767010 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:33.766407    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-w2p8c\""
Sep 14 12:59:33.767010 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:33.766626    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 14 12:59:33.772372 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:33.770275    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp"]
Sep 14 12:59:33.780154 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:33.779830    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:33.782785 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:33.782565    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"]
Sep 14 12:59:33.785102 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:33.785048    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0970c76_9964_4645_b9a9_92aa45a58fc0.slice/crio-b7aac4b9dce701586e75d4ff70871386710f791495c0376ee83dff26b6aeba14 WatchSource:0}: Error finding container b7aac4b9dce701586e75d4ff70871386710f791495c0376ee83dff26b6aeba14: Status 404 returned error can't find the container with id b7aac4b9dce701586e75d4ff70871386710f791495c0376ee83dff26b6aeba14
Sep 14 12:59:33.828127 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:33.828041    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4fc88f14-34e1-4dd9-beaa-7addd2b64afb-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-ccxkp\" (UID: \"4fc88f14-34e1-4dd9-beaa-7addd2b64afb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp"
Sep 14 12:59:33.929232 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:33.929091    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4fc88f14-34e1-4dd9-beaa-7addd2b64afb-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-ccxkp\" (UID: \"4fc88f14-34e1-4dd9-beaa-7addd2b64afb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp"
Sep 14 12:59:33.929516 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:33.929200    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 14 12:59:33.929516 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:33.929396    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc88f14-34e1-4dd9-beaa-7addd2b64afb-tls-certificates podName:4fc88f14-34e1-4dd9-beaa-7addd2b64afb nodeName:}" failed. No retries permitted until 2026-09-14 12:59:34.429373562 +0000 UTC m=+67.720707370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/4fc88f14-34e1-4dd9-beaa-7addd2b64afb-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-ccxkp" (UID: "4fc88f14-34e1-4dd9-beaa-7addd2b64afb") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 14 12:59:34.320616 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:34.320520    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:34.376184 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:34.376097    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:34.434779 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:34.434739    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4fc88f14-34e1-4dd9-beaa-7addd2b64afb-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-ccxkp\" (UID: \"4fc88f14-34e1-4dd9-beaa-7addd2b64afb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp"
Sep 14 12:59:34.434956 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:34.434933    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 14 12:59:34.435020 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:34.435004    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc88f14-34e1-4dd9-beaa-7addd2b64afb-tls-certificates podName:4fc88f14-34e1-4dd9-beaa-7addd2b64afb nodeName:}" failed. No retries permitted until 2026-09-14 12:59:35.434982275 +0000 UTC m=+68.726316075 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/4fc88f14-34e1-4dd9-beaa-7addd2b64afb-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-ccxkp" (UID: "4fc88f14-34e1-4dd9-beaa-7addd2b64afb") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 14 12:59:34.513785 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:34.513739    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:34.655610 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:34.655572    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2bqm" event={"ID":"c32a3bdc-bd26-4891-843d-0808ffa7ec6d","Type":"ContainerStarted","Data":"6e9aebc3d4b9f4da038ab2561cee29d8d68c32ea890229050836f0f722363f8d"}
Sep 14 12:59:34.656253 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:34.656219    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:59:34.658959 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:34.658829    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-49fdc" event={"ID":"f05d48a3-962c-42c0-aba5-1cd6441c74d9","Type":"ContainerStarted","Data":"d88fcae17116202657f1821e8ea4938c10f7e05f4f772b4e3f952f84f3fd72f9"}
Sep 14 12:59:34.661587 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:34.661561    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm" event={"ID":"c0970c76-9964-4645-b9a9-92aa45a58fc0","Type":"ContainerStarted","Data":"98a514124cbffe5eca2c6c01e939dc757ff466d26b50ebd5fecf59ac380f108f"}
Sep 14 12:59:34.661692 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:34.661605    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm" event={"ID":"c0970c76-9964-4645-b9a9-92aa45a58fc0","Type":"ContainerStarted","Data":"b7aac4b9dce701586e75d4ff70871386710f791495c0376ee83dff26b6aeba14"}
Sep 14 12:59:34.661692 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:34.661631    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:34.697161 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:34.696930    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm" podStartSLOduration=2.696905604 podStartE2EDuration="2.696905604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:34.695913092 +0000 UTC m=+67.987246934" watchObservedRunningTime="2026-09-14 12:59:34.696905604 +0000 UTC m=+67.988239423"
Sep 14 12:59:34.697355 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:34.697226    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-g2bqm" podStartSLOduration=33.758524944 podStartE2EDuration="35.697216316s" totalImagesPullingTime="1.938691372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:34.67549562 +0000 UTC m=+67.966829437" watchObservedRunningTime="2026-09-14 12:59:34.697216316 +0000 UTC m=+67.988550133"
Sep 14 12:59:34.714033 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:34.713716    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-49fdc" podStartSLOduration=66.620397164 podStartE2EDuration="1m7.713690548s" totalImagesPullingTime="1.093293384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:34.711427485 +0000 UTC m=+68.002761302" watchObservedRunningTime="2026-09-14 12:59:34.713690548 +0000 UTC m=+68.005024366"
Sep 14 12:59:35.059336 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.059252    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:35.406779 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.406128    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c9fd65b84-rssnp"]
Sep 14 12:59:35.410698 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.410674    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.413459 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.413269    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-pmz8m\""
Sep 14 12:59:35.413459 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.413406    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 14 12:59:35.416218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.415444    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 14 12:59:35.416218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.415698    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 14 12:59:35.416218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.415898    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 14 12:59:35.416218 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.416101    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 14 12:59:35.420760 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.420353    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9fd65b84-rssnp"]
Sep 14 12:59:35.445029 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.444988    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4fc88f14-34e1-4dd9-beaa-7addd2b64afb-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-ccxkp\" (UID: \"4fc88f14-34e1-4dd9-beaa-7addd2b64afb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp"
Sep 14 12:59:35.445187 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.445049    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-service-ca\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.445257 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.445187    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-config\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.445257 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.445221    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv665\" (UniqueName: \"kubernetes.io/projected/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-kube-api-access-mv665\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.445257 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.445249    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-oauth-config\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.445410 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.445294    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-oauth-serving-cert\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.445410 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.445332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-serving-cert\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.448439 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.448413    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4fc88f14-34e1-4dd9-beaa-7addd2b64afb-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-ccxkp\" (UID: \"4fc88f14-34e1-4dd9-beaa-7addd2b64afb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp"
Sep 14 12:59:35.482986 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.482947    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:35.547080 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.546759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-service-ca\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.547080 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.546809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-config\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.547080 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.546843    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv665\" (UniqueName: \"kubernetes.io/projected/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-kube-api-access-mv665\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.547080 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.546871    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-oauth-config\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.547080 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.546912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-oauth-serving-cert\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.547080 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.546954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-serving-cert\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.547843 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.547643    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-config\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.547843 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.547719    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-oauth-serving-cert\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.547992 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.547861    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-service-ca\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.550358 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.550322    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-oauth-config\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.551396 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.551269    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-serving-cert\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.551396 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.551322    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:35.559833 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.559804    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv665\" (UniqueName: \"kubernetes.io/projected/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-kube-api-access-mv665\") pod \"console-c9fd65b84-rssnp\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") " pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:35.585989 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.585723    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp"
Sep 14 12:59:35.724308 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:35.724120    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:38.520501 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:38.520417    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp"]
Sep 14 12:59:38.523196 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:38.523170    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c9fd65b84-rssnp"]
Sep 14 12:59:38.524616 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:38.524586    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc88f14_34e1_4dd9_beaa_7addd2b64afb.slice/crio-9689db7daae9de1c0b4f8943189cc3fed9a65f6fcc44a1a72a17f641c3b02d36 WatchSource:0}: Error finding container 9689db7daae9de1c0b4f8943189cc3fed9a65f6fcc44a1a72a17f641c3b02d36: Status 404 returned error can't find the container with id 9689db7daae9de1c0b4f8943189cc3fed9a65f6fcc44a1a72a17f641c3b02d36
Sep 14 12:59:38.527311 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:38.527261    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20c6e66e_45d2_4989_a74c_6dfedb0cbf0b.slice/crio-70a35cd7a4053c302f4cb27bf208b584c23aed03baecbf800c226496381c6110 WatchSource:0}: Error finding container 70a35cd7a4053c302f4cb27bf208b584c23aed03baecbf800c226496381c6110: Status 404 returned error can't find the container with id 70a35cd7a4053c302f4cb27bf208b584c23aed03baecbf800c226496381c6110
Sep 14 12:59:38.681471 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:38.681430    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6p464" event={"ID":"f9624365-2f56-45f5-ba74-617be3896ac4","Type":"ContainerStarted","Data":"e122461caf905efca5bfd0ddb3cd503b85c922af5afec6c88baff72c9850bc09"}
Sep 14 12:59:38.681677 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:38.681572    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 12:59:38.682957 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:38.682931    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp" event={"ID":"4fc88f14-34e1-4dd9-beaa-7addd2b64afb","Type":"ContainerStarted","Data":"9689db7daae9de1c0b4f8943189cc3fed9a65f6fcc44a1a72a17f641c3b02d36"}
Sep 14 12:59:38.684731 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:38.684686    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dszvl" event={"ID":"db77a7f7-b9e7-47f8-a7d0-524847920a6b","Type":"ContainerStarted","Data":"6fd662ff394122616eba7581ff9c09d6d4d9ceca763aaeb68a80c6a50d4003c1"}
Sep 14 12:59:38.686222 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:38.686190    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9fd65b84-rssnp" event={"ID":"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b","Type":"ContainerStarted","Data":"70a35cd7a4053c302f4cb27bf208b584c23aed03baecbf800c226496381c6110"}
Sep 14 12:59:38.697029 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:38.696988    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6p464" podStartSLOduration=65.826679838 podStartE2EDuration="1m11.696974183s" totalImagesPullingTime="5.870294345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:38.696652681 +0000 UTC m=+71.987986500" watchObservedRunningTime="2026-09-14 12:59:38.696974183 +0000 UTC m=+71.988308001"
Sep 14 12:59:38.712208 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:38.712163    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dszvl" podStartSLOduration=65.836862002 podStartE2EDuration="1m11.71214952s" totalImagesPullingTime="5.875287518s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:38.711202815 +0000 UTC m=+72.002536633" watchObservedRunningTime="2026-09-14 12:59:38.71214952 +0000 UTC m=+72.003483331"
Sep 14 12:59:39.582776 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:39.582727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:39.588408 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:39.588333    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c20fe060-6c64-4048-ac0c-bf214312d5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bnpj9\" (UID: \"c20fe060-6c64-4048-ac0c-bf214312d5ad\") " pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:39.774072 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:39.774011    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-26vf9\""
Sep 14 12:59:39.780272 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:39.780237    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bnpj9"
Sep 14 12:59:40.068454 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:40.068283    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bnpj9"]
Sep 14 12:59:40.071795 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:40.071763    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc20fe060_6c64_4048_ac0c_bf214312d5ad.slice/crio-704e3a0fdc211eaa476d23fe91844c4cb2cd0912038d108d5a5827bb01da2ebe WatchSource:0}: Error finding container 704e3a0fdc211eaa476d23fe91844c4cb2cd0912038d108d5a5827bb01da2ebe: Status 404 returned error can't find the container with id 704e3a0fdc211eaa476d23fe91844c4cb2cd0912038d108d5a5827bb01da2ebe
Sep 14 12:59:40.698246 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:40.698202    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bnpj9" event={"ID":"c20fe060-6c64-4048-ac0c-bf214312d5ad","Type":"ContainerStarted","Data":"fadef25f945fe9f97a92c24bbb6d643dde140a975dd59debaef43a13e0c1ddad"}
Sep 14 12:59:40.698668 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:40.698283    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bnpj9" event={"ID":"c20fe060-6c64-4048-ac0c-bf214312d5ad","Type":"ContainerStarted","Data":"704e3a0fdc211eaa476d23fe91844c4cb2cd0912038d108d5a5827bb01da2ebe"}
Sep 14 12:59:40.700214 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:40.700181    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp" event={"ID":"4fc88f14-34e1-4dd9-beaa-7addd2b64afb","Type":"ContainerStarted","Data":"680b7f43d0ad8c1b507cc2174886bb0bbe5b456862eb6e13929ae05be4282913"}
Sep 14 12:59:40.700512 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:40.700489    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp"
Sep 14 12:59:40.710361 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:40.707864    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp"
Sep 14 12:59:40.735239 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:40.735180    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-ccxkp" podStartSLOduration=6.581453701 podStartE2EDuration="7.735159782s" totalImagesPullingTime="1.153706081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:40.716288356 +0000 UTC m=+74.007622174" watchObservedRunningTime="2026-09-14 12:59:40.735159782 +0000 UTC m=+74.026493599"
Sep 14 12:59:41.776902 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.776823    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:41.818855 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.818817    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"]
Sep 14 12:59:41.825025 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.824995    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:41.827853 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.827580    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 14 12:59:41.827853 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.827832    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 14 12:59:41.828636 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.828610    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 14 12:59:41.828737 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.828646    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-f2npn\""
Sep 14 12:59:41.829409 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.829078    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 14 12:59:41.829409 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.829139    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 14 12:59:41.833933 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.833913    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"]
Sep 14 12:59:41.849882 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.849797    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:41.901471 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.901434    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c798898-5ae9-48a9-a02d-85cbeaa8528b-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-dgrgw\" (UID: \"3c798898-5ae9-48a9-a02d-85cbeaa8528b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:41.901665 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.901491    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f2pj\" (UniqueName: \"kubernetes.io/projected/3c798898-5ae9-48a9-a02d-85cbeaa8528b-kube-api-access-8f2pj\") pod \"prometheus-operator-68bcd7f987-dgrgw\" (UID: \"3c798898-5ae9-48a9-a02d-85cbeaa8528b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:41.901665 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.901598    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c798898-5ae9-48a9-a02d-85cbeaa8528b-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-dgrgw\" (UID: \"3c798898-5ae9-48a9-a02d-85cbeaa8528b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:41.901665 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:41.901641    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c798898-5ae9-48a9-a02d-85cbeaa8528b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-dgrgw\" (UID: \"3c798898-5ae9-48a9-a02d-85cbeaa8528b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:42.002575 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:42.002543    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c798898-5ae9-48a9-a02d-85cbeaa8528b-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-dgrgw\" (UID: \"3c798898-5ae9-48a9-a02d-85cbeaa8528b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:42.002682 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:42.002592    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c798898-5ae9-48a9-a02d-85cbeaa8528b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-dgrgw\" (UID: \"3c798898-5ae9-48a9-a02d-85cbeaa8528b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:42.002682 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:42.002627    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c798898-5ae9-48a9-a02d-85cbeaa8528b-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-dgrgw\" (UID: \"3c798898-5ae9-48a9-a02d-85cbeaa8528b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:42.002682 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:42.002665    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f2pj\" (UniqueName: \"kubernetes.io/projected/3c798898-5ae9-48a9-a02d-85cbeaa8528b-kube-api-access-8f2pj\") pod \"prometheus-operator-68bcd7f987-dgrgw\" (UID: \"3c798898-5ae9-48a9-a02d-85cbeaa8528b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:42.002880 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:42.002859    2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 14 12:59:42.002958 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:42.002923    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c798898-5ae9-48a9-a02d-85cbeaa8528b-prometheus-operator-tls podName:3c798898-5ae9-48a9-a02d-85cbeaa8528b nodeName:}" failed. No retries permitted until 2026-09-14 12:59:42.502905176 +0000 UTC m=+75.794238986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/3c798898-5ae9-48a9-a02d-85cbeaa8528b-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-dgrgw" (UID: "3c798898-5ae9-48a9-a02d-85cbeaa8528b") : secret "prometheus-operator-tls" not found
Sep 14 12:59:42.003515 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:42.003487    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3c798898-5ae9-48a9-a02d-85cbeaa8528b-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-dgrgw\" (UID: \"3c798898-5ae9-48a9-a02d-85cbeaa8528b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:42.005929 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:42.005906    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3c798898-5ae9-48a9-a02d-85cbeaa8528b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-dgrgw\" (UID: \"3c798898-5ae9-48a9-a02d-85cbeaa8528b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:42.010943 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:42.010923    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f2pj\" (UniqueName: \"kubernetes.io/projected/3c798898-5ae9-48a9-a02d-85cbeaa8528b-kube-api-access-8f2pj\") pod \"prometheus-operator-68bcd7f987-dgrgw\" (UID: \"3c798898-5ae9-48a9-a02d-85cbeaa8528b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:42.102406 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:42.102354    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:42.506579 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:42.506524    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c798898-5ae9-48a9-a02d-85cbeaa8528b-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-dgrgw\" (UID: \"3c798898-5ae9-48a9-a02d-85cbeaa8528b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:42.509341 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:42.509315    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c798898-5ae9-48a9-a02d-85cbeaa8528b-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-dgrgw\" (UID: \"3c798898-5ae9-48a9-a02d-85cbeaa8528b\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:42.711166 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:42.711100    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9fd65b84-rssnp" event={"ID":"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b","Type":"ContainerStarted","Data":"e4060dde8ed2725906cdfe29900ee9b3d0963784f8e4678c2b0f9d75d5b61f1e"}
Sep 14 12:59:42.713840 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:42.713804    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bnpj9" event={"ID":"c20fe060-6c64-4048-ac0c-bf214312d5ad","Type":"ContainerStarted","Data":"076f9f11a9c9a7e53c13d47ce1591ae85ead98c61a328f3f9850cb1e0c07761f"}
Sep 14 12:59:42.729017 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:42.728959    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c9fd65b84-rssnp" podStartSLOduration=4.268020785 podStartE2EDuration="7.728941579s" totalImagesPullingTime="3.460920794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:42.728271253 +0000 UTC m=+76.019605075" watchObservedRunningTime="2026-09-14 12:59:42.728941579 +0000 UTC m=+76.020275396"
Sep 14 12:59:42.740561 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:42.740502    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"
Sep 14 12:59:43.032503 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:43.032417    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw"]
Sep 14 12:59:43.036709 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:43.036677    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c798898_5ae9_48a9_a02d_85cbeaa8528b.slice/crio-1b199cbb56f8c6f09a6e68ec576050e7f0435a91361cf38afbd15d9042c2bf64 WatchSource:0}: Error finding container 1b199cbb56f8c6f09a6e68ec576050e7f0435a91361cf38afbd15d9042c2bf64: Status 404 returned error can't find the container with id 1b199cbb56f8c6f09a6e68ec576050e7f0435a91361cf38afbd15d9042c2bf64
Sep 14 12:59:43.189340 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:43.189267    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:43.290506 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:43.290469    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:43.721023 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:43.720985    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bnpj9" event={"ID":"c20fe060-6c64-4048-ac0c-bf214312d5ad","Type":"ContainerStarted","Data":"4402d858a2ea47f6d53964bc787b20fda541651bb56d2234180b2893427edce6"}
Sep 14 12:59:43.723428 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:43.723398    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw" event={"ID":"3c798898-5ae9-48a9-a02d-85cbeaa8528b","Type":"ContainerStarted","Data":"1b199cbb56f8c6f09a6e68ec576050e7f0435a91361cf38afbd15d9042c2bf64"}
Sep 14 12:59:43.741231 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:43.741178    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bnpj9" podStartSLOduration=33.387116806 podStartE2EDuration="36.74116078s" totalImagesPullingTime="3.354043974s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:59:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:43.740137872 +0000 UTC m=+77.031471690" watchObservedRunningTime="2026-09-14 12:59:43.74116078 +0000 UTC m=+77.032494600"
Sep 14 12:59:44.580420 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.580382    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d794f668d-mrl4t"]
Sep 14 12:59:44.583595 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.583564    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.593453 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.593394    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 14 12:59:44.594012 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.593985    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d794f668d-mrl4t"]
Sep 14 12:59:44.624456 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.624423    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-trusted-ca-bundle\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.624659 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.624473    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-config\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.624659 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.624516    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-service-ca\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.624659 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.624573    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-oauth-serving-cert\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.624659 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.624646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-serving-cert\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.624862 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.624685    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r2p7\" (UniqueName: \"kubernetes.io/projected/ec94ce72-4246-4991-bbc6-111d22ab8f83-kube-api-access-7r2p7\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.624862 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.624716    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-oauth-config\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.725226 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.725193    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-trusted-ca-bundle\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.725438 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.725250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-config\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.725438 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.725285    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-service-ca\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.725438 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.725330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-oauth-serving-cert\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.725438 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.725355    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-serving-cert\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.725438 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.725374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r2p7\" (UniqueName: \"kubernetes.io/projected/ec94ce72-4246-4991-bbc6-111d22ab8f83-kube-api-access-7r2p7\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.725438 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.725400    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-oauth-config\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.726112 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.726076    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-config\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.726376 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.726354    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-oauth-serving-cert\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.726473 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.726421    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-service-ca\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.726844 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.726814    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-trusted-ca-bundle\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.728123 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.728101    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-oauth-config\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.729007 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.728986    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-serving-cert\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.734054 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.734016    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r2p7\" (UniqueName: \"kubernetes.io/projected/ec94ce72-4246-4991-bbc6-111d22ab8f83-kube-api-access-7r2p7\") pod \"console-d794f668d-mrl4t\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") " pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:44.895735 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:44.895693    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:45.673292 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:45.673264    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g2bqm"
Sep 14 12:59:45.724934 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:45.724585    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:45.724934 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:45.724648    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:45.730690 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:45.730661    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:45.737954 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:45.737896    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 12:59:48.277874 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:48.277835    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:48.334343 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:48.334308    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:51.756837 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:51.756749    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw" event={"ID":"3c798898-5ae9-48a9-a02d-85cbeaa8528b","Type":"ContainerStarted","Data":"2119789c6b1b5a7c947bf00dc7e70929b5c8902c477ba2f9552d0273ca994e04"}
Sep 14 12:59:51.756837 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:51.756824    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw" event={"ID":"3c798898-5ae9-48a9-a02d-85cbeaa8528b","Type":"ContainerStarted","Data":"ea8594c4aa2de7c4b514533a9bd00a2dc60135754ae55cdda0fa81a9cf2ed434"}
Sep 14 12:59:51.758426 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:51.758392    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-swpmp" event={"ID":"229dfed7-2ade-4f61-8c09-333b42988051","Type":"ContainerStarted","Data":"295ae3e951c9f2445af87ea0af4dff6d8d1ef41c7ef602d677736c4913be98d2"}
Sep 14 12:59:51.758604 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:51.758585    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-swpmp"
Sep 14 12:59:51.763828 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:51.760384    2581 patch_prober.go:28] interesting pod/downloads-76f99584f4-swpmp container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.11:8080/\": dial tcp 10.134.0.11:8080: connect: connection refused" start-of-body=
Sep 14 12:59:51.763828 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:51.760467    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-76f99584f4-swpmp" podUID="229dfed7-2ade-4f61-8c09-333b42988051" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.11:8080/\": dial tcp 10.134.0.11:8080: connect: connection refused"
Sep 14 12:59:51.766654 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:51.766565    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d794f668d-mrl4t"]
Sep 14 12:59:51.770056 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:51.770031    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec94ce72_4246_4991_bbc6_111d22ab8f83.slice/crio-0230f6a098d38607dd92e31aa91c2b35e9ea55e58a06c0fd408bed410238c4c2 WatchSource:0}: Error finding container 0230f6a098d38607dd92e31aa91c2b35e9ea55e58a06c0fd408bed410238c4c2: Status 404 returned error can't find the container with id 0230f6a098d38607dd92e31aa91c2b35e9ea55e58a06c0fd408bed410238c4c2
Sep 14 12:59:51.784923 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:51.784871    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-dgrgw" podStartSLOduration=2.330799194 podStartE2EDuration="10.78485546s" totalImagesPullingTime="8.454056266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:51.783761485 +0000 UTC m=+85.075095319" watchObservedRunningTime="2026-09-14 12:59:51.78485546 +0000 UTC m=+85.076189277"
Sep 14 12:59:51.802408 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:51.802381    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:51.809508 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:51.809452    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-swpmp" podStartSLOduration=1.726577674 podStartE2EDuration="19.809434677s" totalImagesPullingTime="18.082857003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:51.808306069 +0000 UTC m=+85.099639888" watchObservedRunningTime="2026-09-14 12:59:51.809434677 +0000 UTC m=+85.100768493"
Sep 14 12:59:51.998132 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:51.998086    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:52.404028 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:52.403986    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:52.502808 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:52.502773    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:52.765436 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:52.765243    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d794f668d-mrl4t" event={"ID":"ec94ce72-4246-4991-bbc6-111d22ab8f83","Type":"ContainerStarted","Data":"89169d28c4a333ec42e9c9e2c109b0b4253f80f104b17bfc53651b13a84fe0b8"}
Sep 14 12:59:52.765436 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:52.765339    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d794f668d-mrl4t" event={"ID":"ec94ce72-4246-4991-bbc6-111d22ab8f83","Type":"ContainerStarted","Data":"0230f6a098d38607dd92e31aa91c2b35e9ea55e58a06c0fd408bed410238c4c2"}
Sep 14 12:59:52.768237 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:52.768210    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-swpmp"
Sep 14 12:59:52.784682 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:52.784629    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d794f668d-mrl4t" podStartSLOduration=8.784613939 podStartE2EDuration="8.784613939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:52.78348577 +0000 UTC m=+86.074819589" watchObservedRunningTime="2026-09-14 12:59:52.784613939 +0000 UTC m=+86.075947826"
Sep 14 12:59:53.907471 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:53.907439    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:54.007638 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.007602    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:54.117172 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.117133    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:54.140894 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.140854    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"]
Sep 14 12:59:54.170472 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.170384    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"]
Sep 14 12:59:54.170472 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.170448    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"]
Sep 14 12:59:54.170691 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.170494    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"
Sep 14 12:59:54.175550 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.175065    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 14 12:59:54.175550 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.175285    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 14 12:59:54.175550 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.175422    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-r58kl\""
Sep 14 12:59:54.187349 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.187327    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"]
Sep 14 12:59:54.187642 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.187582    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.188173 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.188152    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-62zc6"]
Sep 14 12:59:54.190570 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.190550    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 14 12:59:54.205480 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.205458    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 14 12:59:54.205900 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.205596    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 14 12:59:54.206988 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.206964    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fbk7d\""
Sep 14 12:59:54.213145 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.212960    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:54.218239 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.218215    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.223357 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.223338    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 14 12:59:54.226707 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.226689    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5rv2c\""
Sep 14 12:59:54.227004 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.226991    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 14 12:59:54.227272 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.227258    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 14 12:59:54.302389 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.302343    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:54.307384 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307348    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d2750404-ec81-431c-b230-2dd1fe53560f-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.307545 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1c76a80-a05b-4913-85e3-d85d8174a8c8-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-8gtwj\" (UID: \"c1c76a80-a05b-4913-85e3-d85d8174a8c8\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"
Sep 14 12:59:54.307545 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5g6m\" (UniqueName: \"kubernetes.io/projected/c1c76a80-a05b-4913-85e3-d85d8174a8c8-kube-api-access-t5g6m\") pod \"openshift-state-metrics-67dcf8c979-8gtwj\" (UID: \"c1c76a80-a05b-4913-85e3-d85d8174a8c8\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"
Sep 14 12:59:54.307545 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307473    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1c76a80-a05b-4913-85e3-d85d8174a8c8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-8gtwj\" (UID: \"c1c76a80-a05b-4913-85e3-d85d8174a8c8\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"
Sep 14 12:59:54.307545 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307496    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b443dc84-d2e7-4260-8076-8c0a90d9f275-sys\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.307757 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307569    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b443dc84-d2e7-4260-8076-8c0a90d9f275-metrics-client-ca\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.307757 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307607    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.307757 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b443dc84-d2e7-4260-8076-8c0a90d9f275-root\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.307757 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307674    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2750404-ec81-431c-b230-2dd1fe53560f-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.307757 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307699    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-textfile\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.307757 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307728    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-wtmp\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.308027 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307788    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1c76a80-a05b-4913-85e3-d85d8174a8c8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-8gtwj\" (UID: \"c1c76a80-a05b-4913-85e3-d85d8174a8c8\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"
Sep 14 12:59:54.308027 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307845    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2750404-ec81-431c-b230-2dd1fe53560f-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.308027 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307872    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-accelerators-collector-config\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.308027 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307908    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d2750404-ec81-431c-b230-2dd1fe53560f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.308027 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307941    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2750404-ec81-431c-b230-2dd1fe53560f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.308027 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.307971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vpxb\" (UniqueName: \"kubernetes.io/projected/b443dc84-d2e7-4260-8076-8c0a90d9f275-kube-api-access-7vpxb\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.308293 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.308056    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn4mw\" (UniqueName: \"kubernetes.io/projected/d2750404-ec81-431c-b230-2dd1fe53560f-kube-api-access-vn4mw\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.308293 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.308179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-tls\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.408646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2750404-ec81-431c-b230-2dd1fe53560f-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.408693    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-accelerators-collector-config\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.408724    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d2750404-ec81-431c-b230-2dd1fe53560f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.408755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2750404-ec81-431c-b230-2dd1fe53560f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.408785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vpxb\" (UniqueName: \"kubernetes.io/projected/b443dc84-d2e7-4260-8076-8c0a90d9f275-kube-api-access-7vpxb\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.408817    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn4mw\" (UniqueName: \"kubernetes.io/projected/d2750404-ec81-431c-b230-2dd1fe53560f-kube-api-access-vn4mw\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.408851    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-tls\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.408880    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d2750404-ec81-431c-b230-2dd1fe53560f-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.408906    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1c76a80-a05b-4913-85e3-d85d8174a8c8-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-8gtwj\" (UID: \"c1c76a80-a05b-4913-85e3-d85d8174a8c8\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.408936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5g6m\" (UniqueName: \"kubernetes.io/projected/c1c76a80-a05b-4913-85e3-d85d8174a8c8-kube-api-access-t5g6m\") pod \"openshift-state-metrics-67dcf8c979-8gtwj\" (UID: \"c1c76a80-a05b-4913-85e3-d85d8174a8c8\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.408968    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1c76a80-a05b-4913-85e3-d85d8174a8c8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-8gtwj\" (UID: \"c1c76a80-a05b-4913-85e3-d85d8174a8c8\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.408993    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b443dc84-d2e7-4260-8076-8c0a90d9f275-sys\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.409026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b443dc84-d2e7-4260-8076-8c0a90d9f275-metrics-client-ca\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.409051    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.409080    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b443dc84-d2e7-4260-8076-8c0a90d9f275-root\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.410060 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:54.409085    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 14 12:59:54.411023 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.409108    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2750404-ec81-431c-b230-2dd1fe53560f-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.411023 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.409132    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-textfile\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.411023 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:54.409159    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-tls podName:b443dc84-d2e7-4260-8076-8c0a90d9f275 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:54.909139302 +0000 UTC m=+88.200473114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-tls") pod "node-exporter-62zc6" (UID: "b443dc84-d2e7-4260-8076-8c0a90d9f275") : secret "node-exporter-tls" not found
Sep 14 12:59:54.411023 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.409203    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-wtmp\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.411023 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.409240    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1c76a80-a05b-4913-85e3-d85d8174a8c8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-8gtwj\" (UID: \"c1c76a80-a05b-4913-85e3-d85d8174a8c8\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"
Sep 14 12:59:54.411023 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.409416    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-textfile\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.411023 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.409509    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2750404-ec81-431c-b230-2dd1fe53560f-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.411023 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.409702    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d2750404-ec81-431c-b230-2dd1fe53560f-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.411023 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.409721    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b443dc84-d2e7-4260-8076-8c0a90d9f275-sys\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.411023 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.409779    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b443dc84-d2e7-4260-8076-8c0a90d9f275-root\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.411023 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.409894    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d2750404-ec81-431c-b230-2dd1fe53560f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.411023 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.410036    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-wtmp\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.411023 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.410422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b443dc84-d2e7-4260-8076-8c0a90d9f275-metrics-client-ca\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.411023 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.410680    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1c76a80-a05b-4913-85e3-d85d8174a8c8-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-8gtwj\" (UID: \"c1c76a80-a05b-4913-85e3-d85d8174a8c8\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"
Sep 14 12:59:54.411805 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.411174    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-accelerators-collector-config\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.411805 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.411462    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:54.412750 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.412725    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1c76a80-a05b-4913-85e3-d85d8174a8c8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-8gtwj\" (UID: \"c1c76a80-a05b-4913-85e3-d85d8174a8c8\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"
Sep 14 12:59:54.414452 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.414401    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d2750404-ec81-431c-b230-2dd1fe53560f-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.414829 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.414810    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.415237 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.415213    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2750404-ec81-431c-b230-2dd1fe53560f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.416187 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.416163    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1c76a80-a05b-4913-85e3-d85d8174a8c8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-8gtwj\" (UID: \"c1c76a80-a05b-4913-85e3-d85d8174a8c8\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"
Sep 14 12:59:54.423183 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.423097    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn4mw\" (UniqueName: \"kubernetes.io/projected/d2750404-ec81-431c-b230-2dd1fe53560f-kube-api-access-vn4mw\") pod \"kube-state-metrics-59fbdd77d-95td8\" (UID: \"d2750404-ec81-431c-b230-2dd1fe53560f\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.426255 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.426204    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5g6m\" (UniqueName: \"kubernetes.io/projected/c1c76a80-a05b-4913-85e3-d85d8174a8c8-kube-api-access-t5g6m\") pod \"openshift-state-metrics-67dcf8c979-8gtwj\" (UID: \"c1c76a80-a05b-4913-85e3-d85d8174a8c8\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"
Sep 14 12:59:54.428699 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.428675    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vpxb\" (UniqueName: \"kubernetes.io/projected/b443dc84-d2e7-4260-8076-8c0a90d9f275-kube-api-access-7vpxb\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.485293 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.485248    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"
Sep 14 12:59:54.508313 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.507874    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"
Sep 14 12:59:54.508502 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.508428    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:54.798603 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.798573    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj"]
Sep 14 12:59:54.801332 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:54.801301    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c76a80_a05b_4913_85e3_d85d8174a8c8.slice/crio-f77b5e737d22f77f4752e1151c430fc4bc9e57c0de570b02bc58720d4a33e202 WatchSource:0}: Error finding container f77b5e737d22f77f4752e1151c430fc4bc9e57c0de570b02bc58720d4a33e202: Status 404 returned error can't find the container with id f77b5e737d22f77f4752e1151c430fc4bc9e57c0de570b02bc58720d4a33e202
Sep 14 12:59:54.809323 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.809301    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:54.833392 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.833272    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-95td8"]
Sep 14 12:59:54.836339 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:54.836313    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2750404_ec81_431c_b230_2dd1fe53560f.slice/crio-858e45c2a4245f3ad2c1b581e32f46851f209ba2798ca8dfaa5a945838fe4d36 WatchSource:0}: Error finding container 858e45c2a4245f3ad2c1b581e32f46851f209ba2798ca8dfaa5a945838fe4d36: Status 404 returned error can't find the container with id 858e45c2a4245f3ad2c1b581e32f46851f209ba2798ca8dfaa5a945838fe4d36
Sep 14 12:59:54.896496 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.896458    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:54.896648 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.896524    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:54.902748 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.902514    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:54.915701 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.915674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-tls\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:54.916149 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:54.915841    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 14 12:59:54.916278 ip-10-0-135-35 kubenswrapper[2581]: E0914 12:59:54.916201    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-tls podName:b443dc84-d2e7-4260-8076-8c0a90d9f275 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:55.916179942 +0000 UTC m=+89.207513742 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-tls") pod "node-exporter-62zc6" (UID: "b443dc84-d2e7-4260-8076-8c0a90d9f275") : secret "node-exporter-tls" not found
Sep 14 12:59:54.987924 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:54.987893    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:55.135900 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.135868    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:55.213296 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.213252    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:55.442991 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.442886    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:55.503127 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.503014    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:55.629401 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.629047    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:55.673556 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.672944    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7fcb6fd65f-jkfsm"
Sep 14 12:59:55.780655 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.780486    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8" event={"ID":"d2750404-ec81-431c-b230-2dd1fe53560f","Type":"ContainerStarted","Data":"858e45c2a4245f3ad2c1b581e32f46851f209ba2798ca8dfaa5a945838fe4d36"}
Sep 14 12:59:55.784703 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.784674    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj" event={"ID":"c1c76a80-a05b-4913-85e3-d85d8174a8c8","Type":"ContainerStarted","Data":"89e15324c3213eeb071947bc637aa740e30a4838576e59c0d7b72f81af90b8d2"}
Sep 14 12:59:55.784864 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.784735    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj" event={"ID":"c1c76a80-a05b-4913-85e3-d85d8174a8c8","Type":"ContainerStarted","Data":"24849c216f23d7b3cbdf7ebc52423f5f52610ddf2c6533bbaf7096866b905b04"}
Sep 14 12:59:55.784864 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.784765    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj" event={"ID":"c1c76a80-a05b-4913-85e3-d85d8174a8c8","Type":"ContainerStarted","Data":"f77b5e737d22f77f4752e1151c430fc4bc9e57c0de570b02bc58720d4a33e202"}
Sep 14 12:59:55.790268 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.790086    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 12:59:55.841959 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.841236    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c9fd65b84-rssnp"]
Sep 14 12:59:55.843447 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.843363    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:55.928003 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.926436    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-tls\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:55.934553 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.931113    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b443dc84-d2e7-4260-8076-8c0a90d9f275-node-exporter-tls\") pod \"node-exporter-62zc6\" (UID: \"b443dc84-d2e7-4260-8076-8c0a90d9f275\") " pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:55.939483 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:55.939416    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:56.037933 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.037848    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-62zc6"
Sep 14 12:59:56.040564 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.040510    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:56.136511 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.136467    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:56.140395 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.140363    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7dcd647455-8hnmd"]
Sep 14 12:59:56.160017 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.159940    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7dcd647455-8hnmd"]
Sep 14 12:59:56.160161 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.160071    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.165708 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.163415    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 14 12:59:56.165708 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.163673    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 14 12:59:56.165708 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.163881    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-scwf7\""
Sep 14 12:59:56.165708 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.164188    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 14 12:59:56.165708 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.164544    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 14 12:59:56.165708 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.164641    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2bgn0722h3tre\""
Sep 14 12:59:56.165708 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.164773    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 14 12:59:56.229572 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.229457    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.229572 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.229511    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.229572 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.229566    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mgsr\" (UniqueName: \"kubernetes.io/projected/63dc1702-6042-41a9-96d0-1e623b94ff18-kube-api-access-8mgsr\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.229958 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.229609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-grpc-tls\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.229958 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.229639    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.229958 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.229699    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.229958 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.229751    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63dc1702-6042-41a9-96d0-1e623b94ff18-metrics-client-ca\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.229958 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.229780    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-tls\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.331012 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.330457    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.331012 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.330554    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63dc1702-6042-41a9-96d0-1e623b94ff18-metrics-client-ca\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.331012 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.330584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-tls\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.331012 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.330622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.331012 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.330647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.331012 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.330672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mgsr\" (UniqueName: \"kubernetes.io/projected/63dc1702-6042-41a9-96d0-1e623b94ff18-kube-api-access-8mgsr\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.331012 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.330716    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-grpc-tls\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.331012 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.330749    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.334290 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.334217    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.335997 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.335959    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.336183 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.336146    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-tls\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.336274 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.336151    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-grpc-tls\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.336584 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.336562    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.337197 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.337170    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63dc1702-6042-41a9-96d0-1e623b94ff18-metrics-client-ca\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.337722 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.337673    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/63dc1702-6042-41a9-96d0-1e623b94ff18-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.340264 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.340238    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mgsr\" (UniqueName: \"kubernetes.io/projected/63dc1702-6042-41a9-96d0-1e623b94ff18-kube-api-access-8mgsr\") pod \"thanos-querier-7dcd647455-8hnmd\" (UID: \"63dc1702-6042-41a9-96d0-1e623b94ff18\") " pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.474410 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.474295    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 12:59:56.654770 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.654731    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:56.770352 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:56.770317    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:56.987507 ip-10-0-135-35 kubenswrapper[2581]: W0914 12:59:56.978603    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb443dc84_d2e7_4260_8076_8c0a90d9f275.slice/crio-a1e5b335db910ca18741999dd219e2e696fc053b941ae2e9d13e4e608a23f5f9 WatchSource:0}: Error finding container a1e5b335db910ca18741999dd219e2e696fc053b941ae2e9d13e4e608a23f5f9: Status 404 returned error can't find the container with id a1e5b335db910ca18741999dd219e2e696fc053b941ae2e9d13e4e608a23f5f9
Sep 14 12:59:57.069271 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:57.069241    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:57.158893 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:57.158864    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:57.266313 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:57.265688    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7dcd647455-8hnmd"]
Sep 14 12:59:57.304865 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:57.304829    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:57.361773 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:57.361713    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:57.690923 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:57.690880    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:57.799130 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:57.799088    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd" event={"ID":"63dc1702-6042-41a9-96d0-1e623b94ff18","Type":"ContainerStarted","Data":"c3c241013cfd66a6e4f7dd9e46f0101c36a13e550322e601ea9f822cc2ccfcc7"}
Sep 14 12:59:57.808233 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:57.807226    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8" event={"ID":"d2750404-ec81-431c-b230-2dd1fe53560f","Type":"ContainerStarted","Data":"cf5ca8cc2ada714f056901b75d63af6e0a7b23a272256e9566dda679a1ae19ee"}
Sep 14 12:59:57.808233 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:57.807294    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8" event={"ID":"d2750404-ec81-431c-b230-2dd1fe53560f","Type":"ContainerStarted","Data":"150e0cb88ae7872c3af47254dd1964dc0f0e322fa89e98d8e9efc1ed4bc9cc78"}
Sep 14 12:59:57.808233 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:57.807323    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8" event={"ID":"d2750404-ec81-431c-b230-2dd1fe53560f","Type":"ContainerStarted","Data":"8b71b5e23bb3e83146234259421ff4de8fc050feabbb2dfd5d8dbb6f0e273bcf"}
Sep 14 12:59:57.810191 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:57.810126    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-62zc6" event={"ID":"b443dc84-d2e7-4260-8076-8c0a90d9f275","Type":"ContainerStarted","Data":"a1e5b335db910ca18741999dd219e2e696fc053b941ae2e9d13e4e608a23f5f9"}
Sep 14 12:59:57.817376 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:57.816736    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj" event={"ID":"c1c76a80-a05b-4913-85e3-d85d8174a8c8","Type":"ContainerStarted","Data":"a8a6f662ddbdae00bfbb6b71632623bb9792beda928b1d01492dbeda52c0287f"}
Sep 14 12:59:57.860414 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:57.860365    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-8gtwj" podStartSLOduration=1.908924421 podStartE2EDuration="3.860350003s" totalImagesPullingTime="1.951425582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:57.85943373 +0000 UTC m=+91.150767547" watchObservedRunningTime="2026-09-14 12:59:57.860350003 +0000 UTC m=+91.151683820"
Sep 14 12:59:57.860601 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:57.860478    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-95td8" podStartSLOduration=1.7233689760000002 podStartE2EDuration="3.860470595s" totalImagesPullingTime="2.137101619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:57.837375597 +0000 UTC m=+91.128709415" watchObservedRunningTime="2026-09-14 12:59:57.860470595 +0000 UTC m=+91.151804412"
Sep 14 12:59:58.988500 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:58.988460    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:59.065616 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:59.064424    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:59.184301 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:59.184260    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:59.282958 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:59.282866    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:59.828684 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:59.828646    2581 generic.go:356] "Generic (PLEG): container finished" podID="b443dc84-d2e7-4260-8076-8c0a90d9f275" containerID="74b4dcbbac82c5855df4f720f83e5dbf5db5c5a484e248b73e01b609a3784ddd" exitCode=0
Sep 14 12:59:59.828817 ip-10-0-135-35 kubenswrapper[2581]: I0914 12:59:59.828737    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-62zc6" event={"ID":"b443dc84-d2e7-4260-8076-8c0a90d9f275","Type":"ContainerDied","Data":"74b4dcbbac82c5855df4f720f83e5dbf5db5c5a484e248b73e01b609a3784ddd"}
Sep 14 13:00:00.226314 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.226276    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:00.420918 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.420882    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:00.429386 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.427881    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:00:00.458926 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.458846    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:00:00.459093 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.458988    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.463509 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.463478    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 14 13:00:00.463795 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.463733    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 14 13:00:00.464476 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.464303    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-80k8k0vuvl375\""
Sep 14 13:00:00.464789 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.464767    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 14 13:00:00.467565 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.467155    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 14 13:00:00.467565 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.467382    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 14 13:00:00.467720 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.467656    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 14 13:00:00.468132 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.468016    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 14 13:00:00.468132 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.468109    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 14 13:00:00.470010 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.468475    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 14 13:00:00.470010 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.468722    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 14 13:00:00.470010 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.468936    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 14 13:00:00.470010 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.469462    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 14 13:00:00.470543 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.470507    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 14 13:00:00.470633 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.470569    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 14 13:00:00.471482 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.471462    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9px6r\""
Sep 14 13:00:00.476360 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.476177    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 14 13:00:00.572954 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.572925    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7b724ab-bf18-4452-9802-c262ca05196f-config-out\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573113 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.572961    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d4tn\" (UniqueName: \"kubernetes.io/projected/a7b724ab-bf18-4452-9802-c262ca05196f-kube-api-access-7d4tn\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573113 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.572988    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573113 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573012    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573113 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573067    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573113 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573100    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573284 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573124    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573284 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573284 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573198    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-config\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573416 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573283    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573416 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573315    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573416 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573556 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573556 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573474    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573556 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573697 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573561    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573697 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573598    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573697 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573647    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a7b724ab-bf18-4452-9802-c262ca05196f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573828 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573725    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-web-config\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.573828 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.573767    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.674662 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.674575    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.674662 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.674628    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.674662 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.674661    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.674936 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.674688    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.674936 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.674715    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-config\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.674936 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.674759    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.674936 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.674787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.674936 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.674822    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.674936 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.674856    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.674936 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.674890    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.674936 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.674923    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.675333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.674949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.675333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.674988    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.675333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.675022    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a7b724ab-bf18-4452-9802-c262ca05196f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.675333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.675061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-web-config\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.675333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.675098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.675333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.675144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7b724ab-bf18-4452-9802-c262ca05196f-config-out\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.675333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.675169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d4tn\" (UniqueName: \"kubernetes.io/projected/a7b724ab-bf18-4452-9802-c262ca05196f-kube-api-access-7d4tn\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.675333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.675200    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.675333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.675231    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.676626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.675389    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.676626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.675429    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.676626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.676040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.676626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.676077    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.676626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.676272    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.677337 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.677007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.678720 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.678596    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.680805 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.680302    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7b724ab-bf18-4452-9802-c262ca05196f-config-out\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.680805 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.680739    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.682044 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.682020    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.683095 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.683048    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.683491 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.683469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-web-config\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.684032 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.684010    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a7b724ab-bf18-4452-9802-c262ca05196f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.684188 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.684041    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.684188 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.684139    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.684906 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.684881    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.685103 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.685058    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.685428 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.685405    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-config\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.686253 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.686230    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.687162 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.687137    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d4tn\" (UniqueName: \"kubernetes.io/projected/a7b724ab-bf18-4452-9802-c262ca05196f-kube-api-access-7d4tn\") pod \"prometheus-k8s-0\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.773633 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.773592    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:00.838780 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.838738    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd" event={"ID":"63dc1702-6042-41a9-96d0-1e623b94ff18","Type":"ContainerStarted","Data":"b041236490f5ef5fe9d5cd266b422b4c7f9a037a5861ee68a48f96dadcd7d726"}
Sep 14 13:00:00.838960 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.838829    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd" event={"ID":"63dc1702-6042-41a9-96d0-1e623b94ff18","Type":"ContainerStarted","Data":"444baf13513698b5d3f322b82202764b8c51039abb565653dba40b1b8af22307"}
Sep 14 13:00:00.838960 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.838868    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd" event={"ID":"63dc1702-6042-41a9-96d0-1e623b94ff18","Type":"ContainerStarted","Data":"339b6a57a41289661c8f484887d474a68bbaa211b7622cc533a5aca3a01265ae"}
Sep 14 13:00:00.846059 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.843662    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-62zc6" event={"ID":"b443dc84-d2e7-4260-8076-8c0a90d9f275","Type":"ContainerStarted","Data":"b54c406bfd9505d4ffd25819872b256041089ebbb19c1350fc64c141420a6959"}
Sep 14 13:00:00.846059 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.843736    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-62zc6" event={"ID":"b443dc84-d2e7-4260-8076-8c0a90d9f275","Type":"ContainerStarted","Data":"cbcaabfdf7b7f6e0f923ba71f4c501ac36d84a48613bb5b7a700a3121bd87f37"}
Sep 14 13:00:00.868704 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:00.868641    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-62zc6" podStartSLOduration=4.220514531 podStartE2EDuration="6.868616142s" totalImagesPullingTime="2.561904778s" totalInitContainerRuntime="86.196833ms" isStatefulPod=true podCreationTimestamp="2026-09-14 12:59:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:00:00.861897685 +0000 UTC m=+94.153231503" watchObservedRunningTime="2026-09-14 13:00:00.868616142 +0000 UTC m=+94.159949960"
Sep 14 13:00:01.080064 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:01.079987    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:00:01.083413 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:00:01.083381    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b724ab_bf18_4452_9802_c262ca05196f.slice/crio-691dc83cbc8ba0d9f1cc6043fe553c20ec0176be663aafd5af866fed47b50126 WatchSource:0}: Error finding container 691dc83cbc8ba0d9f1cc6043fe553c20ec0176be663aafd5af866fed47b50126: Status 404 returned error can't find the container with id 691dc83cbc8ba0d9f1cc6043fe553c20ec0176be663aafd5af866fed47b50126
Sep 14 13:00:01.854573 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:01.854520    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd" event={"ID":"63dc1702-6042-41a9-96d0-1e623b94ff18","Type":"ContainerStarted","Data":"06a2bccd5c2f1639ae273b34e890a27c9931ca8106e86e5aaf7e273ade21bb48"}
Sep 14 13:00:01.855089 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:01.854650    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 13:00:01.855089 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:01.854692    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd" event={"ID":"63dc1702-6042-41a9-96d0-1e623b94ff18","Type":"ContainerStarted","Data":"152bbb4c5c6696d505e972f18636e693c5c45684668e2e68e01714ee3d0e54ac"}
Sep 14 13:00:01.855089 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:01.854731    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd" event={"ID":"63dc1702-6042-41a9-96d0-1e623b94ff18","Type":"ContainerStarted","Data":"da3673861bef2185ea412944f983f713b3e00e126f11335cc5095ff75b0fb880"}
Sep 14 13:00:01.855828 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:01.855803    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerStarted","Data":"691dc83cbc8ba0d9f1cc6043fe553c20ec0176be663aafd5af866fed47b50126"}
Sep 14 13:00:01.877617 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:01.877516    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd" podStartSLOduration=2.142754726 podStartE2EDuration="5.87750169s" totalImagesPullingTime="3.734746964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:00:01.876892639 +0000 UTC m=+95.168226455" watchObservedRunningTime="2026-09-14 13:00:01.87750169 +0000 UTC m=+95.168835507"
Sep 14 13:00:02.558015 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:02.557981    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:02.980653 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:02.980571    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:03.865524 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:03.865488    2581 generic.go:356] "Generic (PLEG): container finished" podID="a7b724ab-bf18-4452-9802-c262ca05196f" containerID="1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079" exitCode=0
Sep 14 13:00:03.865718 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:03.865559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerDied","Data":"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"}
Sep 14 13:00:05.768734 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:05.768690    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:05.824496 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:05.824457    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:06.562027 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:06.561993    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:06.663480 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:06.663445    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:07.868252 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:07.868216    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7dcd647455-8hnmd"
Sep 14 13:00:08.890779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:08.890746    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerStarted","Data":"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"}
Sep 14 13:00:08.891103 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:08.890843    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerStarted","Data":"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"}
Sep 14 13:00:08.891103 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:08.890871    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerStarted","Data":"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"}
Sep 14 13:00:08.891103 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:08.890894    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerStarted","Data":"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"}
Sep 14 13:00:09.410350 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:09.410312    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d794f668d-mrl4t"]
Sep 14 13:00:09.508993 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:09.508961    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:09.620446 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:09.620410    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:09.682808 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:09.682734    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:09.692942 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:09.692916    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6p464"
Sep 14 13:00:09.898575 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:09.898517    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerStarted","Data":"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"}
Sep 14 13:00:09.899030 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:09.898610    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerStarted","Data":"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"}
Sep 14 13:00:09.929268 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:09.929201    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.338507007 podStartE2EDuration="9.929177712s" totalImagesPullingTime="6.567300199s" totalInitContainerRuntime="23.370506ms" isStatefulPod=false podCreationTimestamp="2026-09-14 13:00:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:00:09.92655535 +0000 UTC m=+103.217889164" watchObservedRunningTime="2026-09-14 13:00:09.929177712 +0000 UTC m=+103.220511529"
Sep 14 13:00:10.127857 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:10.127829    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:10.709683 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:10.709652    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:10.773951 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:10.773916    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:12.863198 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:12.863158    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:12.938148 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:12.938115    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:12.997402 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:12.997370    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:17.123822 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:17.123788    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:20.878508 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:20.878465    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c9fd65b84-rssnp" podUID="20c6e66e-45d2-4989-a74c-6dfedb0cbf0b" containerName="console" containerID="cri-o://e4060dde8ed2725906cdfe29900ee9b3d0963784f8e4678c2b0f9d75d5b61f1e" gracePeriod=15
Sep 14 13:00:21.147484 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.147398    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 13:00:21.156439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.156416    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-service-ca\") pod \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") "
Sep 14 13:00:21.156577 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.156451    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-oauth-serving-cert\") pod \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") "
Sep 14 13:00:21.156577 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.156476    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-oauth-config\") pod \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") "
Sep 14 13:00:21.156701 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.156603    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-config\") pod \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") "
Sep 14 13:00:21.156701 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.156682    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mv665\" (UniqueName: \"kubernetes.io/projected/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-kube-api-access-mv665\") pod \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") "
Sep 14 13:00:21.156804 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.156724    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-serving-cert\") pod \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\" (UID: \"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b\") "
Sep 14 13:00:21.156917 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.156892    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "20c6e66e-45d2-4989-a74c-6dfedb0cbf0b" (UID: "20c6e66e-45d2-4989-a74c-6dfedb0cbf0b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:00:21.157000 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.156916    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-service-ca" (OuterVolumeSpecName: "service-ca") pod "20c6e66e-45d2-4989-a74c-6dfedb0cbf0b" (UID: "20c6e66e-45d2-4989-a74c-6dfedb0cbf0b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:00:21.157000 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.156981    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-config" (OuterVolumeSpecName: "console-config") pod "20c6e66e-45d2-4989-a74c-6dfedb0cbf0b" (UID: "20c6e66e-45d2-4989-a74c-6dfedb0cbf0b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:00:21.157101 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.156997    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-service-ca\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:00:21.157101 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.157017    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-oauth-serving-cert\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:00:21.158842 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.158817    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "20c6e66e-45d2-4989-a74c-6dfedb0cbf0b" (UID: "20c6e66e-45d2-4989-a74c-6dfedb0cbf0b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:00:21.158928 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.158891    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "20c6e66e-45d2-4989-a74c-6dfedb0cbf0b" (UID: "20c6e66e-45d2-4989-a74c-6dfedb0cbf0b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:00:21.159108 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.159079    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-kube-api-access-mv665" (OuterVolumeSpecName: "kube-api-access-mv665") pod "20c6e66e-45d2-4989-a74c-6dfedb0cbf0b" (UID: "20c6e66e-45d2-4989-a74c-6dfedb0cbf0b"). InnerVolumeSpecName "kube-api-access-mv665". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:00:21.258106 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.258082    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-oauth-config\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:00:21.258106 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.258104    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-config\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:00:21.258267 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.258114    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mv665\" (UniqueName: \"kubernetes.io/projected/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-kube-api-access-mv665\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:00:21.258267 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.258132    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b-console-serving-cert\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:00:21.946704 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.946670    2581 generic.go:356] "Generic (PLEG): container finished" podID="20c6e66e-45d2-4989-a74c-6dfedb0cbf0b" containerID="e4060dde8ed2725906cdfe29900ee9b3d0963784f8e4678c2b0f9d75d5b61f1e" exitCode=0
Sep 14 13:00:21.947115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.946715    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c9fd65b84-rssnp"
Sep 14 13:00:21.947115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.946764    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9fd65b84-rssnp" event={"ID":"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b","Type":"ContainerDied","Data":"e4060dde8ed2725906cdfe29900ee9b3d0963784f8e4678c2b0f9d75d5b61f1e"}
Sep 14 13:00:21.947115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.946823    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c9fd65b84-rssnp" event={"ID":"20c6e66e-45d2-4989-a74c-6dfedb0cbf0b","Type":"ContainerDied","Data":"70a35cd7a4053c302f4cb27bf208b584c23aed03baecbf800c226496381c6110"}
Sep 14 13:00:21.947115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.946846    2581 scope.go:117] "RemoveContainer" containerID="e4060dde8ed2725906cdfe29900ee9b3d0963784f8e4678c2b0f9d75d5b61f1e"
Sep 14 13:00:21.955744 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.955727    2581 scope.go:117] "RemoveContainer" containerID="e4060dde8ed2725906cdfe29900ee9b3d0963784f8e4678c2b0f9d75d5b61f1e"
Sep 14 13:00:21.956009 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:00:21.955991    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4060dde8ed2725906cdfe29900ee9b3d0963784f8e4678c2b0f9d75d5b61f1e\": container with ID starting with e4060dde8ed2725906cdfe29900ee9b3d0963784f8e4678c2b0f9d75d5b61f1e not found: ID does not exist" containerID="e4060dde8ed2725906cdfe29900ee9b3d0963784f8e4678c2b0f9d75d5b61f1e"
Sep 14 13:00:21.956071 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.956017    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4060dde8ed2725906cdfe29900ee9b3d0963784f8e4678c2b0f9d75d5b61f1e"} err="failed to get container status \"e4060dde8ed2725906cdfe29900ee9b3d0963784f8e4678c2b0f9d75d5b61f1e\": rpc error: code = NotFound desc = could not find container \"e4060dde8ed2725906cdfe29900ee9b3d0963784f8e4678c2b0f9d75d5b61f1e\": container with ID starting with e4060dde8ed2725906cdfe29900ee9b3d0963784f8e4678c2b0f9d75d5b61f1e not found: ID does not exist"
Sep 14 13:00:21.963602 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.963579    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c9fd65b84-rssnp"]
Sep 14 13:00:21.967952 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:21.967929    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c9fd65b84-rssnp"]
Sep 14 13:00:23.245459 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:23.245423    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20c6e66e-45d2-4989-a74c-6dfedb0cbf0b" path="/var/lib/kubelet/pods/20c6e66e-45d2-4989-a74c-6dfedb0cbf0b/volumes"
Sep 14 13:00:23.269460 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:23.269399    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xf7sp_9cec680f-1687-4f62-af5d-43a9c8f6f630/serve-healthcheck-canary/0.log"
Sep 14 13:00:25.130956 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:25.130916    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:25.193815 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:25.193741    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:31.652795 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:31.652757    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:31.754316 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:31.754281    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:31.808584 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:31.808551    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:34.430566 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.430500    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d794f668d-mrl4t" podUID="ec94ce72-4246-4991-bbc6-111d22ab8f83" containerName="console" containerID="cri-o://89169d28c4a333ec42e9c9e2c109b0b4253f80f104b17bfc53651b13a84fe0b8" gracePeriod=15
Sep 14 13:00:34.733643 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.733609    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 13:00:34.781253 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.781224    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-trusted-ca-bundle\") pod \"ec94ce72-4246-4991-bbc6-111d22ab8f83\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") "
Sep 14 13:00:34.781253 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.781262    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7r2p7\" (UniqueName: \"kubernetes.io/projected/ec94ce72-4246-4991-bbc6-111d22ab8f83-kube-api-access-7r2p7\") pod \"ec94ce72-4246-4991-bbc6-111d22ab8f83\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") "
Sep 14 13:00:34.781494 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.781306    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-config\") pod \"ec94ce72-4246-4991-bbc6-111d22ab8f83\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") "
Sep 14 13:00:34.781494 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.781339    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-service-ca\") pod \"ec94ce72-4246-4991-bbc6-111d22ab8f83\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") "
Sep 14 13:00:34.781494 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.781386    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-oauth-serving-cert\") pod \"ec94ce72-4246-4991-bbc6-111d22ab8f83\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") "
Sep 14 13:00:34.781667 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.781634    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ec94ce72-4246-4991-bbc6-111d22ab8f83" (UID: "ec94ce72-4246-4991-bbc6-111d22ab8f83"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:00:34.781848 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.781808    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ec94ce72-4246-4991-bbc6-111d22ab8f83" (UID: "ec94ce72-4246-4991-bbc6-111d22ab8f83"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:00:34.781848 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.781835    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-serving-cert\") pod \"ec94ce72-4246-4991-bbc6-111d22ab8f83\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") "
Sep 14 13:00:34.782042 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.781856    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-config" (OuterVolumeSpecName: "console-config") pod "ec94ce72-4246-4991-bbc6-111d22ab8f83" (UID: "ec94ce72-4246-4991-bbc6-111d22ab8f83"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:00:34.782042 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.781889    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-service-ca" (OuterVolumeSpecName: "service-ca") pod "ec94ce72-4246-4991-bbc6-111d22ab8f83" (UID: "ec94ce72-4246-4991-bbc6-111d22ab8f83"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:00:34.782042 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.781907    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-oauth-config\") pod \"ec94ce72-4246-4991-bbc6-111d22ab8f83\" (UID: \"ec94ce72-4246-4991-bbc6-111d22ab8f83\") "
Sep 14 13:00:34.782218 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.782149    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-config\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:00:34.782218 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.782169    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-service-ca\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:00:34.782218 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.782181    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-oauth-serving-cert\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:00:34.782218 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.782196    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec94ce72-4246-4991-bbc6-111d22ab8f83-trusted-ca-bundle\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:00:34.783715 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.783685    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec94ce72-4246-4991-bbc6-111d22ab8f83-kube-api-access-7r2p7" (OuterVolumeSpecName: "kube-api-access-7r2p7") pod "ec94ce72-4246-4991-bbc6-111d22ab8f83" (UID: "ec94ce72-4246-4991-bbc6-111d22ab8f83"). InnerVolumeSpecName "kube-api-access-7r2p7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:00:34.783872 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.783854    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ec94ce72-4246-4991-bbc6-111d22ab8f83" (UID: "ec94ce72-4246-4991-bbc6-111d22ab8f83"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:00:34.784025 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.783999    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ec94ce72-4246-4991-bbc6-111d22ab8f83" (UID: "ec94ce72-4246-4991-bbc6-111d22ab8f83"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:00:34.882890 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.882856    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-oauth-config\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:00:34.882890 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.882888    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7r2p7\" (UniqueName: \"kubernetes.io/projected/ec94ce72-4246-4991-bbc6-111d22ab8f83-kube-api-access-7r2p7\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:00:34.883077 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:34.882903    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec94ce72-4246-4991-bbc6-111d22ab8f83-console-serving-cert\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:00:35.001768 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:35.001683    2581 generic.go:356] "Generic (PLEG): container finished" podID="ec94ce72-4246-4991-bbc6-111d22ab8f83" containerID="89169d28c4a333ec42e9c9e2c109b0b4253f80f104b17bfc53651b13a84fe0b8" exitCode=0
Sep 14 13:00:35.001768 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:35.001733    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d794f668d-mrl4t"
Sep 14 13:00:35.001967 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:35.001762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d794f668d-mrl4t" event={"ID":"ec94ce72-4246-4991-bbc6-111d22ab8f83","Type":"ContainerDied","Data":"89169d28c4a333ec42e9c9e2c109b0b4253f80f104b17bfc53651b13a84fe0b8"}
Sep 14 13:00:35.001967 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:35.001849    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d794f668d-mrl4t" event={"ID":"ec94ce72-4246-4991-bbc6-111d22ab8f83","Type":"ContainerDied","Data":"0230f6a098d38607dd92e31aa91c2b35e9ea55e58a06c0fd408bed410238c4c2"}
Sep 14 13:00:35.001967 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:35.001872    2581 scope.go:117] "RemoveContainer" containerID="89169d28c4a333ec42e9c9e2c109b0b4253f80f104b17bfc53651b13a84fe0b8"
Sep 14 13:00:35.014208 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:35.013975    2581 scope.go:117] "RemoveContainer" containerID="89169d28c4a333ec42e9c9e2c109b0b4253f80f104b17bfc53651b13a84fe0b8"
Sep 14 13:00:35.014470 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:00:35.014363    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89169d28c4a333ec42e9c9e2c109b0b4253f80f104b17bfc53651b13a84fe0b8\": container with ID starting with 89169d28c4a333ec42e9c9e2c109b0b4253f80f104b17bfc53651b13a84fe0b8 not found: ID does not exist" containerID="89169d28c4a333ec42e9c9e2c109b0b4253f80f104b17bfc53651b13a84fe0b8"
Sep 14 13:00:35.014470 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:35.014397    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89169d28c4a333ec42e9c9e2c109b0b4253f80f104b17bfc53651b13a84fe0b8"} err="failed to get container status \"89169d28c4a333ec42e9c9e2c109b0b4253f80f104b17bfc53651b13a84fe0b8\": rpc error: code = NotFound desc = could not find container \"89169d28c4a333ec42e9c9e2c109b0b4253f80f104b17bfc53651b13a84fe0b8\": container with ID starting with 89169d28c4a333ec42e9c9e2c109b0b4253f80f104b17bfc53651b13a84fe0b8 not found: ID does not exist"
Sep 14 13:00:35.028358 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:35.028331    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d794f668d-mrl4t"]
Sep 14 13:00:35.031042 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:35.031019    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d794f668d-mrl4t"]
Sep 14 13:00:35.245588 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:35.245558    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec94ce72-4246-4991-bbc6-111d22ab8f83" path="/var/lib/kubelet/pods/ec94ce72-4246-4991-bbc6-111d22ab8f83/volumes"
Sep 14 13:00:35.249073 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:00:35.249045    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:00.745336 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:00.745297    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:00.774212 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:00.774183    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:00.793366 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:00.793336    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:01.117590 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:01.117557    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:13.416277 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:13.416227    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:13.485378 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:13.485320    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:18.695947 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:18.695909    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:01:18.696389 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:18.696200    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="prometheus" containerID="cri-o://73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa" gracePeriod=600
Sep 14 13:01:18.696389 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:18.696243    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="kube-rbac-proxy" containerID="cri-o://c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd" gracePeriod=600
Sep 14 13:01:18.696389 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:18.696276    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471" gracePeriod=600
Sep 14 13:01:18.696389 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:18.696259    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="thanos-sidecar" containerID="cri-o://82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7" gracePeriod=600
Sep 14 13:01:18.696389 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:18.696265    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="kube-rbac-proxy-web" containerID="cri-o://d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764" gracePeriod=600
Sep 14 13:01:18.696672 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:18.696396    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="config-reloader" containerID="cri-o://00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59" gracePeriod=600
Sep 14 13:01:18.955193 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:18.955135    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.053892 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.053859    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-web-config\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054071 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.053906    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-config\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054071 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.053925    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-db\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054071 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.053946    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-metrics-client-certs\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054071 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.053965    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-kube-rbac-proxy\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054253 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054114    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054253 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054185    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-thanos-prometheus-http-client-file\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054253 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054221    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-kubelet-serving-ca-bundle\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054253 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054249    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-0\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054456 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054281    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-serving-certs-ca-bundle\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054456 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054316    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-metrics-client-ca\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054456 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054342    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-1\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054456 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054375    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-grpc-tls\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054456 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054409    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a7b724ab-bf18-4452-9802-c262ca05196f-tls-assets\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054456 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054442    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054771 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054490    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-2\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054771 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054522    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-trusted-ca-bundle\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054771 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054584    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7b724ab-bf18-4452-9802-c262ca05196f-config-out\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054771 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054615    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7d4tn\" (UniqueName: \"kubernetes.io/projected/a7b724ab-bf18-4452-9802-c262ca05196f-kube-api-access-7d4tn\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054771 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054641    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-tls\") pod \"a7b724ab-bf18-4452-9802-c262ca05196f\" (UID: \"a7b724ab-bf18-4452-9802-c262ca05196f\") "
Sep 14 13:01:19.054771 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054746    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:01:19.055054 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.054958    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.055054 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.055002    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:01:19.056309 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.055569    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:01:19.056603 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.056553    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:01:19.056733 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.056632    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:01:19.057330 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.057302    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:01:19.057753 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.057579    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:19.057753 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.057600    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-config" (OuterVolumeSpecName: "config") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:19.057753 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.057650    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:19.057753 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.057693    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:19.057753 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.057729    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:19.058462 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.058433    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:01:19.058748 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.058702    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:19.058960 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.058937    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b724ab-bf18-4452-9802-c262ca05196f-config-out" (OuterVolumeSpecName: "config-out") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:01:19.059037 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.058960    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:01:19.059152 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.059130    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:19.059240 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.059222    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b724ab-bf18-4452-9802-c262ca05196f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:01:19.059290 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.059226    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:19.060342 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.060308    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b724ab-bf18-4452-9802-c262ca05196f-kube-api-access-7d4tn" (OuterVolumeSpecName: "kube-api-access-7d4tn") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "kube-api-access-7d4tn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:01:19.068047 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.068022    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-web-config" (OuterVolumeSpecName: "web-config") pod "a7b724ab-bf18-4452-9802-c262ca05196f" (UID: "a7b724ab-bf18-4452-9802-c262ca05196f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:19.156183 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156127    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156183 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156175    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a7b724ab-bf18-4452-9802-c262ca05196f-config-out\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156183 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156186    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7d4tn\" (UniqueName: \"kubernetes.io/projected/a7b724ab-bf18-4452-9802-c262ca05196f-kube-api-access-7d4tn\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156183 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156195    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156183 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156206    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-web-config\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156215    2581 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-config\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156223    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-db\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156232    2581 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-metrics-client-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156240    2581 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-kube-rbac-proxy\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156250    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156259    2581 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156267    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156276    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156285    2581 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-configmap-metrics-client-ca\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156294    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156303    2581 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-grpc-tls\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156312    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a7b724ab-bf18-4452-9802-c262ca05196f-tls-assets\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156320    2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a7b724ab-bf18-4452-9802-c262ca05196f-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.156487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.156327    2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a7b724ab-bf18-4452-9802-c262ca05196f-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:01:19.174199 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174166    2581 generic.go:356] "Generic (PLEG): container finished" podID="a7b724ab-bf18-4452-9802-c262ca05196f" containerID="5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471" exitCode=0
Sep 14 13:01:19.174199 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174195    2581 generic.go:356] "Generic (PLEG): container finished" podID="a7b724ab-bf18-4452-9802-c262ca05196f" containerID="c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd" exitCode=0
Sep 14 13:01:19.174199 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174203    2581 generic.go:356] "Generic (PLEG): container finished" podID="a7b724ab-bf18-4452-9802-c262ca05196f" containerID="d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764" exitCode=0
Sep 14 13:01:19.174439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174211    2581 generic.go:356] "Generic (PLEG): container finished" podID="a7b724ab-bf18-4452-9802-c262ca05196f" containerID="82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7" exitCode=0
Sep 14 13:01:19.174439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174219    2581 generic.go:356] "Generic (PLEG): container finished" podID="a7b724ab-bf18-4452-9802-c262ca05196f" containerID="00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59" exitCode=0
Sep 14 13:01:19.174439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174226    2581 generic.go:356] "Generic (PLEG): container finished" podID="a7b724ab-bf18-4452-9802-c262ca05196f" containerID="73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa" exitCode=0
Sep 14 13:01:19.174439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174249    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.174439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174244    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerDied","Data":"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"}
Sep 14 13:01:19.174439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174412    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerDied","Data":"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"}
Sep 14 13:01:19.174699 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174442    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerDied","Data":"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"}
Sep 14 13:01:19.174699 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174468    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerDied","Data":"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"}
Sep 14 13:01:19.174699 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174485    2581 scope.go:117] "RemoveContainer" containerID="5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"
Sep 14 13:01:19.174699 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174493    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerDied","Data":"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"}
Sep 14 13:01:19.174699 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174520    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerDied","Data":"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"}
Sep 14 13:01:19.174699 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.174557    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a7b724ab-bf18-4452-9802-c262ca05196f","Type":"ContainerDied","Data":"691dc83cbc8ba0d9f1cc6043fe553c20ec0176be663aafd5af866fed47b50126"}
Sep 14 13:01:19.183204 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.183187    2581 scope.go:117] "RemoveContainer" containerID="c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"
Sep 14 13:01:19.190553 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.190513    2581 scope.go:117] "RemoveContainer" containerID="d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"
Sep 14 13:01:19.197996 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.197976    2581 scope.go:117] "RemoveContainer" containerID="82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"
Sep 14 13:01:19.198753 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.198732    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:01:19.202229 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.202206    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:01:19.205409 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.205389    2581 scope.go:117] "RemoveContainer" containerID="00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"
Sep 14 13:01:19.214101 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.214082    2581 scope.go:117] "RemoveContainer" containerID="73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"
Sep 14 13:01:19.221420 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.221402    2581 scope.go:117] "RemoveContainer" containerID="1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"
Sep 14 13:01:19.227662 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.227639    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:01:19.228112 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228057    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20c6e66e-45d2-4989-a74c-6dfedb0cbf0b" containerName="console"
Sep 14 13:01:19.228165 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228116    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="20c6e66e-45d2-4989-a74c-6dfedb0cbf0b" containerName="console"
Sep 14 13:01:19.228165 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228130    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="kube-rbac-proxy"
Sep 14 13:01:19.228165 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228139    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="kube-rbac-proxy"
Sep 14 13:01:19.228412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228182    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="prometheus"
Sep 14 13:01:19.228412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228215    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="prometheus"
Sep 14 13:01:19.228412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228284    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="init-config-reloader"
Sep 14 13:01:19.228412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228293    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="init-config-reloader"
Sep 14 13:01:19.228412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228307    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="thanos-sidecar"
Sep 14 13:01:19.228412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228316    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="thanos-sidecar"
Sep 14 13:01:19.228412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228330    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="kube-rbac-proxy-thanos"
Sep 14 13:01:19.228412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228338    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="kube-rbac-proxy-thanos"
Sep 14 13:01:19.228412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228349    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec94ce72-4246-4991-bbc6-111d22ab8f83" containerName="console"
Sep 14 13:01:19.228412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228356    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec94ce72-4246-4991-bbc6-111d22ab8f83" containerName="console"
Sep 14 13:01:19.228412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228366    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="config-reloader"
Sep 14 13:01:19.228412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228375    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="config-reloader"
Sep 14 13:01:19.228412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228389    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="kube-rbac-proxy-web"
Sep 14 13:01:19.228412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228398    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="kube-rbac-proxy-web"
Sep 14 13:01:19.229197 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228500    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="kube-rbac-proxy-web"
Sep 14 13:01:19.229197 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228513    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec94ce72-4246-4991-bbc6-111d22ab8f83" containerName="console"
Sep 14 13:01:19.229197 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228523    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="config-reloader"
Sep 14 13:01:19.229197 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228554    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="prometheus"
Sep 14 13:01:19.229197 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228564    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="20c6e66e-45d2-4989-a74c-6dfedb0cbf0b" containerName="console"
Sep 14 13:01:19.229197 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228573    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="kube-rbac-proxy"
Sep 14 13:01:19.229197 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228581    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="thanos-sidecar"
Sep 14 13:01:19.229197 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.228590    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" containerName="kube-rbac-proxy-thanos"
Sep 14 13:01:19.229773 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.229753    2581 scope.go:117] "RemoveContainer" containerID="5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"
Sep 14 13:01:19.230060 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:01:19.230043    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471\": container with ID starting with 5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471 not found: ID does not exist" containerID="5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"
Sep 14 13:01:19.230103 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.230070    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"} err="failed to get container status \"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471\": rpc error: code = NotFound desc = could not find container \"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471\": container with ID starting with 5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471 not found: ID does not exist"
Sep 14 13:01:19.230103 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.230089    2581 scope.go:117] "RemoveContainer" containerID="c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"
Sep 14 13:01:19.230330 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:01:19.230311    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd\": container with ID starting with c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd not found: ID does not exist" containerID="c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"
Sep 14 13:01:19.230375 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.230336    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"} err="failed to get container status \"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd\": rpc error: code = NotFound desc = could not find container \"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd\": container with ID starting with c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd not found: ID does not exist"
Sep 14 13:01:19.230375 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.230349    2581 scope.go:117] "RemoveContainer" containerID="d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"
Sep 14 13:01:19.230616 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:01:19.230593    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764\": container with ID starting with d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764 not found: ID does not exist" containerID="d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"
Sep 14 13:01:19.230758 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.230620    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"} err="failed to get container status \"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764\": rpc error: code = NotFound desc = could not find container \"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764\": container with ID starting with d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764 not found: ID does not exist"
Sep 14 13:01:19.230758 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.230636    2581 scope.go:117] "RemoveContainer" containerID="82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"
Sep 14 13:01:19.230895 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:01:19.230877    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7\": container with ID starting with 82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7 not found: ID does not exist" containerID="82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"
Sep 14 13:01:19.230953 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.230903    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"} err="failed to get container status \"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7\": rpc error: code = NotFound desc = could not find container \"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7\": container with ID starting with 82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7 not found: ID does not exist"
Sep 14 13:01:19.230953 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.230925    2581 scope.go:117] "RemoveContainer" containerID="00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"
Sep 14 13:01:19.231201 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:01:19.231185    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59\": container with ID starting with 00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59 not found: ID does not exist" containerID="00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"
Sep 14 13:01:19.231243 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.231206    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"} err="failed to get container status \"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59\": rpc error: code = NotFound desc = could not find container \"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59\": container with ID starting with 00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59 not found: ID does not exist"
Sep 14 13:01:19.231243 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.231220    2581 scope.go:117] "RemoveContainer" containerID="73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"
Sep 14 13:01:19.231450 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:01:19.231432    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa\": container with ID starting with 73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa not found: ID does not exist" containerID="73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"
Sep 14 13:01:19.231518 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.231456    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"} err="failed to get container status \"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa\": rpc error: code = NotFound desc = could not find container \"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa\": container with ID starting with 73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa not found: ID does not exist"
Sep 14 13:01:19.231518 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.231475    2581 scope.go:117] "RemoveContainer" containerID="1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"
Sep 14 13:01:19.231767 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:01:19.231750    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079\": container with ID starting with 1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079 not found: ID does not exist" containerID="1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"
Sep 14 13:01:19.231848 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.231773    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"} err="failed to get container status \"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079\": rpc error: code = NotFound desc = could not find container \"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079\": container with ID starting with 1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079 not found: ID does not exist"
Sep 14 13:01:19.231848 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.231791    2581 scope.go:117] "RemoveContainer" containerID="5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"
Sep 14 13:01:19.232065 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.232041    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"} err="failed to get container status \"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471\": rpc error: code = NotFound desc = could not find container \"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471\": container with ID starting with 5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471 not found: ID does not exist"
Sep 14 13:01:19.232065 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.232061    2581 scope.go:117] "RemoveContainer" containerID="c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"
Sep 14 13:01:19.232333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.232315    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"} err="failed to get container status \"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd\": rpc error: code = NotFound desc = could not find container \"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd\": container with ID starting with c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd not found: ID does not exist"
Sep 14 13:01:19.232333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.232332    2581 scope.go:117] "RemoveContainer" containerID="d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"
Sep 14 13:01:19.232615 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.232571    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"} err="failed to get container status \"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764\": rpc error: code = NotFound desc = could not find container \"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764\": container with ID starting with d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764 not found: ID does not exist"
Sep 14 13:01:19.232615 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.232595    2581 scope.go:117] "RemoveContainer" containerID="82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"
Sep 14 13:01:19.232770 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.232755    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.232824 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.232803    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"} err="failed to get container status \"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7\": rpc error: code = NotFound desc = could not find container \"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7\": container with ID starting with 82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7 not found: ID does not exist"
Sep 14 13:01:19.232947 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.232826    2581 scope.go:117] "RemoveContainer" containerID="00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"
Sep 14 13:01:19.233192 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.233096    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"} err="failed to get container status \"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59\": rpc error: code = NotFound desc = could not find container \"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59\": container with ID starting with 00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59 not found: ID does not exist"
Sep 14 13:01:19.233192 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.233125    2581 scope.go:117] "RemoveContainer" containerID="73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"
Sep 14 13:01:19.233443 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.233422    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"} err="failed to get container status \"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa\": rpc error: code = NotFound desc = could not find container \"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa\": container with ID starting with 73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa not found: ID does not exist"
Sep 14 13:01:19.233513 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.233444    2581 scope.go:117] "RemoveContainer" containerID="1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"
Sep 14 13:01:19.233712 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.233694    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"} err="failed to get container status \"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079\": rpc error: code = NotFound desc = could not find container \"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079\": container with ID starting with 1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079 not found: ID does not exist"
Sep 14 13:01:19.233773 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.233713    2581 scope.go:117] "RemoveContainer" containerID="5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"
Sep 14 13:01:19.233953 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.233932    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"} err="failed to get container status \"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471\": rpc error: code = NotFound desc = could not find container \"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471\": container with ID starting with 5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471 not found: ID does not exist"
Sep 14 13:01:19.234036 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.233954    2581 scope.go:117] "RemoveContainer" containerID="c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"
Sep 14 13:01:19.234197 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.234179    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"} err="failed to get container status \"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd\": rpc error: code = NotFound desc = could not find container \"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd\": container with ID starting with c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd not found: ID does not exist"
Sep 14 13:01:19.234248 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.234197    2581 scope.go:117] "RemoveContainer" containerID="d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"
Sep 14 13:01:19.234465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.234446    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"} err="failed to get container status \"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764\": rpc error: code = NotFound desc = could not find container \"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764\": container with ID starting with d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764 not found: ID does not exist"
Sep 14 13:01:19.234512 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.234465    2581 scope.go:117] "RemoveContainer" containerID="82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"
Sep 14 13:01:19.234716 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.234700    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"} err="failed to get container status \"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7\": rpc error: code = NotFound desc = could not find container \"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7\": container with ID starting with 82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7 not found: ID does not exist"
Sep 14 13:01:19.234760 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.234716    2581 scope.go:117] "RemoveContainer" containerID="00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"
Sep 14 13:01:19.234958 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.234943    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"} err="failed to get container status \"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59\": rpc error: code = NotFound desc = could not find container \"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59\": container with ID starting with 00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59 not found: ID does not exist"
Sep 14 13:01:19.235009 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.234957    2581 scope.go:117] "RemoveContainer" containerID="73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"
Sep 14 13:01:19.235159 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.235142    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"} err="failed to get container status \"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa\": rpc error: code = NotFound desc = could not find container \"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa\": container with ID starting with 73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa not found: ID does not exist"
Sep 14 13:01:19.235201 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.235159    2581 scope.go:117] "RemoveContainer" containerID="1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"
Sep 14 13:01:19.235422 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.235403    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"} err="failed to get container status \"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079\": rpc error: code = NotFound desc = could not find container \"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079\": container with ID starting with 1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079 not found: ID does not exist"
Sep 14 13:01:19.235422 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.235421    2581 scope.go:117] "RemoveContainer" containerID="5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"
Sep 14 13:01:19.235624 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.235606    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 14 13:01:19.235688 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.235673    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"} err="failed to get container status \"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471\": rpc error: code = NotFound desc = could not find container \"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471\": container with ID starting with 5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471 not found: ID does not exist"
Sep 14 13:01:19.235739 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.235690    2581 scope.go:117] "RemoveContainer" containerID="c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"
Sep 14 13:01:19.235980 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.235956    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"} err="failed to get container status \"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd\": rpc error: code = NotFound desc = could not find container \"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd\": container with ID starting with c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd not found: ID does not exist"
Sep 14 13:01:19.236082 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.235981    2581 scope.go:117] "RemoveContainer" containerID="d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"
Sep 14 13:01:19.236082 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236025    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 14 13:01:19.236082 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236075    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 14 13:01:19.236240 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236075    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 14 13:01:19.236295 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236245    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"} err="failed to get container status \"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764\": rpc error: code = NotFound desc = could not find container \"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764\": container with ID starting with d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764 not found: ID does not exist"
Sep 14 13:01:19.236295 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236270    2581 scope.go:117] "RemoveContainer" containerID="82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"
Sep 14 13:01:19.236399 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236380    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 14 13:01:19.236454 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236402    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 14 13:01:19.236454 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236384    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 14 13:01:19.236454 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236417    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 14 13:01:19.236628 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236556    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"} err="failed to get container status \"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7\": rpc error: code = NotFound desc = could not find container \"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7\": container with ID starting with 82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7 not found: ID does not exist"
Sep 14 13:01:19.236628 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236580    2581 scope.go:117] "RemoveContainer" containerID="00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"
Sep 14 13:01:19.236628 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236606    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 14 13:01:19.236781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236672    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9px6r\""
Sep 14 13:01:19.236781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236754    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 14 13:01:19.236881 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236848    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"} err="failed to get container status \"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59\": rpc error: code = NotFound desc = could not find container \"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59\": container with ID starting with 00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59 not found: ID does not exist"
Sep 14 13:01:19.236881 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236871    2581 scope.go:117] "RemoveContainer" containerID="73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"
Sep 14 13:01:19.236988 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236916    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 14 13:01:19.236988 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.236922    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 14 13:01:19.237164 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.237141    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"} err="failed to get container status \"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa\": rpc error: code = NotFound desc = could not find container \"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa\": container with ID starting with 73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa not found: ID does not exist"
Sep 14 13:01:19.237164 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.237164    2581 scope.go:117] "RemoveContainer" containerID="1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"
Sep 14 13:01:19.237426 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.237404    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-80k8k0vuvl375\""
Sep 14 13:01:19.237426 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.237424    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 14 13:01:19.237623 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.237438    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"} err="failed to get container status \"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079\": rpc error: code = NotFound desc = could not find container \"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079\": container with ID starting with 1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079 not found: ID does not exist"
Sep 14 13:01:19.237623 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.237460    2581 scope.go:117] "RemoveContainer" containerID="5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"
Sep 14 13:01:19.237771 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.237750    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"} err="failed to get container status \"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471\": rpc error: code = NotFound desc = could not find container \"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471\": container with ID starting with 5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471 not found: ID does not exist"
Sep 14 13:01:19.237838 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.237773    2581 scope.go:117] "RemoveContainer" containerID="c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"
Sep 14 13:01:19.238082 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.238056    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"} err="failed to get container status \"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd\": rpc error: code = NotFound desc = could not find container \"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd\": container with ID starting with c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd not found: ID does not exist"
Sep 14 13:01:19.238180 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.238082    2581 scope.go:117] "RemoveContainer" containerID="d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"
Sep 14 13:01:19.238337 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.238320    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"} err="failed to get container status \"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764\": rpc error: code = NotFound desc = could not find container \"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764\": container with ID starting with d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764 not found: ID does not exist"
Sep 14 13:01:19.238401 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.238338    2581 scope.go:117] "RemoveContainer" containerID="82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"
Sep 14 13:01:19.238605 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.238585    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"} err="failed to get container status \"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7\": rpc error: code = NotFound desc = could not find container \"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7\": container with ID starting with 82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7 not found: ID does not exist"
Sep 14 13:01:19.238695 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.238607    2581 scope.go:117] "RemoveContainer" containerID="00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"
Sep 14 13:01:19.238851 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.238834    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"} err="failed to get container status \"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59\": rpc error: code = NotFound desc = could not find container \"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59\": container with ID starting with 00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59 not found: ID does not exist"
Sep 14 13:01:19.238900 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.238851    2581 scope.go:117] "RemoveContainer" containerID="73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"
Sep 14 13:01:19.239080 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.239057    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"} err="failed to get container status \"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa\": rpc error: code = NotFound desc = could not find container \"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa\": container with ID starting with 73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa not found: ID does not exist"
Sep 14 13:01:19.239160 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.239082    2581 scope.go:117] "RemoveContainer" containerID="1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"
Sep 14 13:01:19.239288 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.239272    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"} err="failed to get container status \"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079\": rpc error: code = NotFound desc = could not find container \"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079\": container with ID starting with 1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079 not found: ID does not exist"
Sep 14 13:01:19.239288 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.239287    2581 scope.go:117] "RemoveContainer" containerID="5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"
Sep 14 13:01:19.239582 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.239559    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471"} err="failed to get container status \"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471\": rpc error: code = NotFound desc = could not find container \"5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471\": container with ID starting with 5916cd61b096e11e41faec9bedb15245cf0d79579101463deca5430847801471 not found: ID does not exist"
Sep 14 13:01:19.239664 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.239583    2581 scope.go:117] "RemoveContainer" containerID="c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"
Sep 14 13:01:19.239939 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.239849    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd"} err="failed to get container status \"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd\": rpc error: code = NotFound desc = could not find container \"c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd\": container with ID starting with c792d6471d7c086e08d2a061183ab36dacbc8f33a7d356cf4a47b51e14da22dd not found: ID does not exist"
Sep 14 13:01:19.239939 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.239877    2581 scope.go:117] "RemoveContainer" containerID="d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"
Sep 14 13:01:19.240186 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.240147    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764"} err="failed to get container status \"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764\": rpc error: code = NotFound desc = could not find container \"d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764\": container with ID starting with d496be9f533cf60a275cdbd92f8baed9d47f3cecb1fcfa83eaf030220abdb764 not found: ID does not exist"
Sep 14 13:01:19.240186 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.240169    2581 scope.go:117] "RemoveContainer" containerID="82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"
Sep 14 13:01:19.240395 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.240372    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7"} err="failed to get container status \"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7\": rpc error: code = NotFound desc = could not find container \"82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7\": container with ID starting with 82fbfa511004b79d44e4311aee60a0d1403cd9557fc789fbf2d27b99c289c0c7 not found: ID does not exist"
Sep 14 13:01:19.240395 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.240395    2581 scope.go:117] "RemoveContainer" containerID="00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"
Sep 14 13:01:19.240672 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.240651    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59"} err="failed to get container status \"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59\": rpc error: code = NotFound desc = could not find container \"00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59\": container with ID starting with 00ee53f1df1b37eebcd6003a597750f19b701a46432bb786ba1c345dbd7abf59 not found: ID does not exist"
Sep 14 13:01:19.240760 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.240674    2581 scope.go:117] "RemoveContainer" containerID="73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"
Sep 14 13:01:19.241040 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.240972    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa"} err="failed to get container status \"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa\": rpc error: code = NotFound desc = could not find container \"73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa\": container with ID starting with 73adaf78213921817daae6dcda4457df83d74d012ca905b8d06fc5147cf3b3aa not found: ID does not exist"
Sep 14 13:01:19.241040 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.240999    2581 scope.go:117] "RemoveContainer" containerID="1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"
Sep 14 13:01:19.241259 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.241240    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079"} err="failed to get container status \"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079\": rpc error: code = NotFound desc = could not find container \"1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079\": container with ID starting with 1ed96389c5ad54211ba4e503c1d8736cac4c7911bc1f10a2b0f57832e21d9079 not found: ID does not exist"
Sep 14 13:01:19.241471 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.241453    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 14 13:01:19.243643 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.243622    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 14 13:01:19.248997 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.248973    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7b724ab-bf18-4452-9802-c262ca05196f" path="/var/lib/kubelet/pods/a7b724ab-bf18-4452-9802-c262ca05196f/volumes"
Sep 14 13:01:19.249892 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.249873    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:01:19.257508 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257482    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.257613 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257556    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.257613 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257598    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.257700 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257635    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.257700 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257664    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.257700 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257692    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.257801 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257718    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-config-out\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.257801 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257742    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.257801 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257766    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-web-config\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.257904 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257829    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.257904 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257854    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd2g7\" (UniqueName: \"kubernetes.io/projected/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-kube-api-access-kd2g7\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.257904 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257874    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.257904 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257888    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.258034 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257906    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.258034 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257975    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.258034 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.257999    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.258034 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.258021    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.258163 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.258050    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-config\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.258163 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.258066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.258163 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.258085    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359016 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.358950    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359016 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.358990    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359214 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359111    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359214 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359214 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359178    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359214 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-config-out\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359410 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359410 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359258    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-web-config\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359410 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359410 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359320    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd2g7\" (UniqueName: \"kubernetes.io/projected/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-kube-api-access-kd2g7\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359410 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359410 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359410 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359392    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359426    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359458    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359495    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359516    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-config\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359574    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.359781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359637    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.360226 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359819    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.360226 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.359836    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.360226 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.360061    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.360664 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.360637    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.360760 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.360731    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.362037 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.361996    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.362430 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.362210    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.362430 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.362394    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-config-out\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.362430 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.362409    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-config\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.362869 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.362844    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.363058 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.363031    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-web-config\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.363967 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.363942    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.364070 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.363965    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.364245 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.364226    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.364721 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.364700    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.364813 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.364764    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.364813 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.364767    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.365059 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.365039    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.369209 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.369191    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd2g7\" (UniqueName: \"kubernetes.io/projected/e64b2c7b-8c6e-48ff-9b8b-15a813581b97-kube-api-access-kd2g7\") pod \"prometheus-k8s-0\" (UID: \"e64b2c7b-8c6e-48ff-9b8b-15a813581b97\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.545251 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.545166    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:19.826322 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:19.826246    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:01:19.829868 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:01:19.829838    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode64b2c7b_8c6e_48ff_9b8b_15a813581b97.slice/crio-d477641cb4b740bdb810c46bc1fcc406c0df4b16d0a38a953e06eaf36816771b WatchSource:0}: Error finding container d477641cb4b740bdb810c46bc1fcc406c0df4b16d0a38a953e06eaf36816771b: Status 404 returned error can't find the container with id d477641cb4b740bdb810c46bc1fcc406c0df4b16d0a38a953e06eaf36816771b
Sep 14 13:01:20.180180 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:20.180147    2581 generic.go:356] "Generic (PLEG): container finished" podID="e64b2c7b-8c6e-48ff-9b8b-15a813581b97" containerID="15c94cc2a348c690ba4f5b14462b8911c07d09337ee0fd4ba7eb9080f4d50c7a" exitCode=0
Sep 14 13:01:20.180413 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:20.180234    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e64b2c7b-8c6e-48ff-9b8b-15a813581b97","Type":"ContainerDied","Data":"15c94cc2a348c690ba4f5b14462b8911c07d09337ee0fd4ba7eb9080f4d50c7a"}
Sep 14 13:01:20.180413 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:20.180332    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e64b2c7b-8c6e-48ff-9b8b-15a813581b97","Type":"ContainerStarted","Data":"d477641cb4b740bdb810c46bc1fcc406c0df4b16d0a38a953e06eaf36816771b"}
Sep 14 13:01:21.189043 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:21.189002    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e64b2c7b-8c6e-48ff-9b8b-15a813581b97","Type":"ContainerStarted","Data":"c9e612b868d1e99c87ef4baafbb2135ac28854c6a21f851729b63f7a2b4aa472"}
Sep 14 13:01:21.189428 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:21.189072    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e64b2c7b-8c6e-48ff-9b8b-15a813581b97","Type":"ContainerStarted","Data":"a040ce71b90444f4d172d74d67243239f80b8f82a66fef3bdff9fe69cd4ee6fc"}
Sep 14 13:01:21.189428 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:21.189103    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e64b2c7b-8c6e-48ff-9b8b-15a813581b97","Type":"ContainerStarted","Data":"3c4e78da54a36d8b4d575ad02138feb48fab32ce99489fb0983252c7f9f4c7b7"}
Sep 14 13:01:21.189428 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:21.189129    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e64b2c7b-8c6e-48ff-9b8b-15a813581b97","Type":"ContainerStarted","Data":"64eea1b55e3b9bc183764442a77f956e9f6e271fee79ad5c934e3d41b12df2ce"}
Sep 14 13:01:21.189428 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:21.189154    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e64b2c7b-8c6e-48ff-9b8b-15a813581b97","Type":"ContainerStarted","Data":"bf00cc0cb1e00a08a983e943eda0e581756f5edc80a93eb656679dba21ae28f1"}
Sep 14 13:01:21.189428 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:21.189176    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e64b2c7b-8c6e-48ff-9b8b-15a813581b97","Type":"ContainerStarted","Data":"c576ab466a748c03c948861fae13aca52fd54bf473908d2dcb11032a073eef8b"}
Sep 14 13:01:21.219073 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:21.219006    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.1994745509999998 podStartE2EDuration="2.218988052s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.513501ms" isStatefulPod=false podCreationTimestamp="2026-09-14 13:01:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:01:21.216886278 +0000 UTC m=+174.508220123" watchObservedRunningTime="2026-09-14 13:01:21.218988052 +0000 UTC m=+174.510321869"
Sep 14 13:01:21.844663 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:21.844628    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:21.902845 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:21.902811    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:21.985770 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:21.985740    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:22.087010 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:22.086978    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:22.646688 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:22.646652    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:24.545714 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:24.545672    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:32.918982 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:32.918947    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:33.019895 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:33.019861    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:33.074737 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:33.074703    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:39.481034 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:39.480998    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:53.264182 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:53.264129    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:53.319537 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:01:53.319502    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:19.545745 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:19.545688    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:02:19.561667 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:19.561640    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:02:20.428720 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:20.428694    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:02:28.571232 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:28.571195    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:28.634054 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:28.634006    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:33.252226 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:33.252190    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:33.315771 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:33.315743    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:33.405523 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:33.405495    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:33.485248 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:33.485198    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:33.989252 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:33.989218    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:40.618651 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:40.618603    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:40.677644 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:40.677590    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:44.192680 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:44.192644    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:44.293508 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:44.293475    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:44.367454 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:44.367419    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:55.147400 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:55.147317    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:55.245866 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:55.245836    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:55.491950 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:02:55.491869    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:03:27.119288 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:03:27.119248    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:03:27.120999 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:03:27.120977    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:03:27.125046 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:03:27.125027    2581 kubelet.go:1648] "Image garbage collection succeeded"
Sep 14 13:03:52.437414 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:03:52.437372    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:03:53.803898 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:03:53.803856    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:12.639749 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:12.639714    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:12.739700 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:12.739667    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:14.397636 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:14.397597    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:15.241077 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:15.241040    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:18.242054 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:18.241981    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:18.343826 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:18.343789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:19.841728 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:19.841626    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:19.951822 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:19.951788    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:20.492461 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:20.492426    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:20.593581 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:20.593523    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:22.885948 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:22.885910    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:23.085969 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:23.085907    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:25.209085 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:25.209038    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:25.272664 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:25.272620    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:29.256543 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.256509    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j5sfp"]
Sep 14 13:04:29.259882 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.259864    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-j5sfp"
Sep 14 13:04:29.262425 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.262399    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 14 13:04:29.262565 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.262470    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-qct44\""
Sep 14 13:04:29.263384 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.263368    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 14 13:04:29.267752 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.267728    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j5sfp"]
Sep 14 13:04:29.352889 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.352849    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9f5b3680-da50-4c16-acfc-803125eb3e6c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j5sfp\" (UID: \"9f5b3680-da50-4c16-acfc-803125eb3e6c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j5sfp"
Sep 14 13:04:29.353063 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.352997    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s2jf\" (UniqueName: \"kubernetes.io/projected/9f5b3680-da50-4c16-acfc-803125eb3e6c-kube-api-access-5s2jf\") pod \"cert-manager-webhook-6486ffd997-j5sfp\" (UID: \"9f5b3680-da50-4c16-acfc-803125eb3e6c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j5sfp"
Sep 14 13:04:29.454290 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.454251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s2jf\" (UniqueName: \"kubernetes.io/projected/9f5b3680-da50-4c16-acfc-803125eb3e6c-kube-api-access-5s2jf\") pod \"cert-manager-webhook-6486ffd997-j5sfp\" (UID: \"9f5b3680-da50-4c16-acfc-803125eb3e6c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j5sfp"
Sep 14 13:04:29.454436 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.454305    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9f5b3680-da50-4c16-acfc-803125eb3e6c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j5sfp\" (UID: \"9f5b3680-da50-4c16-acfc-803125eb3e6c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j5sfp"
Sep 14 13:04:29.465137 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.465103    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9f5b3680-da50-4c16-acfc-803125eb3e6c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j5sfp\" (UID: \"9f5b3680-da50-4c16-acfc-803125eb3e6c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j5sfp"
Sep 14 13:04:29.465376 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.465355    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s2jf\" (UniqueName: \"kubernetes.io/projected/9f5b3680-da50-4c16-acfc-803125eb3e6c-kube-api-access-5s2jf\") pod \"cert-manager-webhook-6486ffd997-j5sfp\" (UID: \"9f5b3680-da50-4c16-acfc-803125eb3e6c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j5sfp"
Sep 14 13:04:29.585746 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.585665    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-j5sfp"
Sep 14 13:04:29.858256 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.858221    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j5sfp"]
Sep 14 13:04:29.861487 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:04:29.861456    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f5b3680_da50_4c16_acfc_803125eb3e6c.slice/crio-19805f6d5ba043d4173c872a146d846ceb8db344daba4cc92268eebd93fd915e WatchSource:0}: Error finding container 19805f6d5ba043d4173c872a146d846ceb8db344daba4cc92268eebd93fd915e: Status 404 returned error can't find the container with id 19805f6d5ba043d4173c872a146d846ceb8db344daba4cc92268eebd93fd915e
Sep 14 13:04:29.863396 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.863380    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 13:04:29.913763 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:29.913728    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-j5sfp" event={"ID":"9f5b3680-da50-4c16-acfc-803125eb3e6c","Type":"ContainerStarted","Data":"19805f6d5ba043d4173c872a146d846ceb8db344daba4cc92268eebd93fd915e"}
Sep 14 13:04:30.629705 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:30.629666    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:30.729012 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:30.728976    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:30.891234 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:30.891150    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:32.929061 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:32.929024    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-j5sfp" event={"ID":"9f5b3680-da50-4c16-acfc-803125eb3e6c","Type":"ContainerStarted","Data":"7d220953793286391f78600a82a60e71941a0bef9a34ad39c0083c8d32912de8"}
Sep 14 13:04:32.929444 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:32.929146    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-j5sfp"
Sep 14 13:04:32.946891 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:32.946838    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-j5sfp" podStartSLOduration=1.07128839 podStartE2EDuration="3.946820756s" totalImagesPullingTime="2.875532366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:04:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:04:32.946549732 +0000 UTC m=+366.237883543" watchObservedRunningTime="2026-09-14 13:04:32.946820756 +0000 UTC m=+366.238154573"
Sep 14 13:04:32.996306 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:32.996237    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:33.110836 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:33.110806    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:38.320392 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:38.320360    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:38.419972 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:38.419943    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:38.936068 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:38.936038    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-j5sfp"
Sep 14 13:04:39.897387 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:39.897355    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:39.997952 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:39.997913    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:40.339742 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:40.339665    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:50.771858 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:50.771813    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:50.879012 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:50.878977    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:50.971728 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:50.971692    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:52.013005 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.012978    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:52.110884 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.110846    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:52.605610 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.605574    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:52.690649 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.690616    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:52.740601 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.740568    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"]
Sep 14 13:04:52.744409 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.744388    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:52.748664 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.748637    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 14 13:04:52.748781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.748715    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-64cdb\""
Sep 14 13:04:52.748781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.748719    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 14 13:04:52.749080 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.749050    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 14 13:04:52.749198 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.749051    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 14 13:04:52.749198 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.749183    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 14 13:04:52.749752 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.749734    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/360df0fb-ef47-4edd-83fb-24abdf61f92d-metrics-cert\") pod \"lws-controller-manager-6d5f86f746-p8fmq\" (UID: \"360df0fb-ef47-4edd-83fb-24abdf61f92d\") " pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:52.749816 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.749774    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/360df0fb-ef47-4edd-83fb-24abdf61f92d-manager-config\") pod \"lws-controller-manager-6d5f86f746-p8fmq\" (UID: \"360df0fb-ef47-4edd-83fb-24abdf61f92d\") " pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:52.749816 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.749794    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/360df0fb-ef47-4edd-83fb-24abdf61f92d-cert\") pod \"lws-controller-manager-6d5f86f746-p8fmq\" (UID: \"360df0fb-ef47-4edd-83fb-24abdf61f92d\") " pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:52.749912 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.749817    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsgg8\" (UniqueName: \"kubernetes.io/projected/360df0fb-ef47-4edd-83fb-24abdf61f92d-kube-api-access-qsgg8\") pod \"lws-controller-manager-6d5f86f746-p8fmq\" (UID: \"360df0fb-ef47-4edd-83fb-24abdf61f92d\") " pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:52.754288 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.754266    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"]
Sep 14 13:04:52.791898 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.791873    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:52.850281 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.850242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/360df0fb-ef47-4edd-83fb-24abdf61f92d-metrics-cert\") pod \"lws-controller-manager-6d5f86f746-p8fmq\" (UID: \"360df0fb-ef47-4edd-83fb-24abdf61f92d\") " pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:52.850443 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.850296    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/360df0fb-ef47-4edd-83fb-24abdf61f92d-manager-config\") pod \"lws-controller-manager-6d5f86f746-p8fmq\" (UID: \"360df0fb-ef47-4edd-83fb-24abdf61f92d\") " pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:52.850443 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.850323    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/360df0fb-ef47-4edd-83fb-24abdf61f92d-cert\") pod \"lws-controller-manager-6d5f86f746-p8fmq\" (UID: \"360df0fb-ef47-4edd-83fb-24abdf61f92d\") " pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:52.850443 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.850344    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsgg8\" (UniqueName: \"kubernetes.io/projected/360df0fb-ef47-4edd-83fb-24abdf61f92d-kube-api-access-qsgg8\") pod \"lws-controller-manager-6d5f86f746-p8fmq\" (UID: \"360df0fb-ef47-4edd-83fb-24abdf61f92d\") " pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:52.851066 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.851038    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/360df0fb-ef47-4edd-83fb-24abdf61f92d-manager-config\") pod \"lws-controller-manager-6d5f86f746-p8fmq\" (UID: \"360df0fb-ef47-4edd-83fb-24abdf61f92d\") " pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:52.852762 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.852742    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/360df0fb-ef47-4edd-83fb-24abdf61f92d-metrics-cert\") pod \"lws-controller-manager-6d5f86f746-p8fmq\" (UID: \"360df0fb-ef47-4edd-83fb-24abdf61f92d\") " pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:52.852853 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.852775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/360df0fb-ef47-4edd-83fb-24abdf61f92d-cert\") pod \"lws-controller-manager-6d5f86f746-p8fmq\" (UID: \"360df0fb-ef47-4edd-83fb-24abdf61f92d\") " pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:52.875398 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.875325    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsgg8\" (UniqueName: \"kubernetes.io/projected/360df0fb-ef47-4edd-83fb-24abdf61f92d-kube-api-access-qsgg8\") pod \"lws-controller-manager-6d5f86f746-p8fmq\" (UID: \"360df0fb-ef47-4edd-83fb-24abdf61f92d\") " pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:52.928570 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:52.928524    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:53.039144 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.039108    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:53.054285 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.054254    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:53.086075 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.086039    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"]
Sep 14 13:04:53.088567 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.088550    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.092300 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.092271    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 14 13:04:53.093138 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.093115    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 14 13:04:53.093269 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.093154    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 14 13:04:53.094485 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.094304    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 14 13:04:53.094624 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.094552    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bdfph\""
Sep 14 13:04:53.113730 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.113440    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"]
Sep 14 13:04:53.152384 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.152303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/527f4585-b998-46ec-a189-ff50c371c3a9-tmp\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.152384 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.152335    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/527f4585-b998-46ec-a189-ff50c371c3a9-webhook-cert\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.152384 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.152363    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vmw8\" (UniqueName: \"kubernetes.io/projected/527f4585-b998-46ec-a189-ff50c371c3a9-kube-api-access-9vmw8\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.152614 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.152388    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/527f4585-b998-46ec-a189-ff50c371c3a9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.152614 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.152441    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/527f4585-b998-46ec-a189-ff50c371c3a9-manifests\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.240407 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.240375    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:53.252917 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.252883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/527f4585-b998-46ec-a189-ff50c371c3a9-tmp\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.253081 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.252926    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/527f4585-b998-46ec-a189-ff50c371c3a9-webhook-cert\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.253081 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.253040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vmw8\" (UniqueName: \"kubernetes.io/projected/527f4585-b998-46ec-a189-ff50c371c3a9-kube-api-access-9vmw8\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.253200 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.253083    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/527f4585-b998-46ec-a189-ff50c371c3a9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.253200 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.253136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/527f4585-b998-46ec-a189-ff50c371c3a9-manifests\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.253511 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.253482    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/527f4585-b998-46ec-a189-ff50c371c3a9-manifests\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.255102 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.255073    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/527f4585-b998-46ec-a189-ff50c371c3a9-tmp\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.255326 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.255308    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/527f4585-b998-46ec-a189-ff50c371c3a9-webhook-cert\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.255367 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.255341    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/527f4585-b998-46ec-a189-ff50c371c3a9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.266690 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.266667    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vmw8\" (UniqueName: \"kubernetes.io/projected/527f4585-b998-46ec-a189-ff50c371c3a9-kube-api-access-9vmw8\") pod \"opendatahub-operator-controller-manager-697cf97c89-mwdgg\" (UID: \"527f4585-b998-46ec-a189-ff50c371c3a9\") " pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.332603 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.332575    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"]
Sep 14 13:04:53.334984 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.334951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:53.335363 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:04:53.335341    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod360df0fb_ef47_4edd_83fb_24abdf61f92d.slice/crio-a83fc105eb48c4a5462b0831d7fb240ab59c84928824fb9757d34dad45cdebac WatchSource:0}: Error finding container a83fc105eb48c4a5462b0831d7fb240ab59c84928824fb9757d34dad45cdebac: Status 404 returned error can't find the container with id a83fc105eb48c4a5462b0831d7fb240ab59c84928824fb9757d34dad45cdebac
Sep 14 13:04:53.401490 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.401458    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:04:53.431170 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.431144    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:53.531599 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.531557    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:53.631570 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.631511    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:53.674873 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:53.674846    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"]
Sep 14 13:04:53.677315 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:04:53.677289    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod527f4585_b998_46ec_a189_ff50c371c3a9.slice/crio-9600ae6a6b45b878dc2c9f93b831d0c0f5f2ccae9b9a4d0243e2536c71b1e62d WatchSource:0}: Error finding container 9600ae6a6b45b878dc2c9f93b831d0c0f5f2ccae9b9a4d0243e2536c71b1e62d: Status 404 returned error can't find the container with id 9600ae6a6b45b878dc2c9f93b831d0c0f5f2ccae9b9a4d0243e2536c71b1e62d
Sep 14 13:04:54.014010 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:54.013906    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg" event={"ID":"527f4585-b998-46ec-a189-ff50c371c3a9","Type":"ContainerStarted","Data":"9600ae6a6b45b878dc2c9f93b831d0c0f5f2ccae9b9a4d0243e2536c71b1e62d"}
Sep 14 13:04:54.015082 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:54.015054    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq" event={"ID":"360df0fb-ef47-4edd-83fb-24abdf61f92d","Type":"ContainerStarted","Data":"a83fc105eb48c4a5462b0831d7fb240ab59c84928824fb9757d34dad45cdebac"}
Sep 14 13:04:58.036079 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:58.035980    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq" event={"ID":"360df0fb-ef47-4edd-83fb-24abdf61f92d","Type":"ContainerStarted","Data":"733b8cec40f109e9a3c806f195b4ca1e56bbe837275297796678e518732a5b0e"}
Sep 14 13:04:58.036548 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:58.036133    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:04:58.037661 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:58.037637    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg" event={"ID":"527f4585-b998-46ec-a189-ff50c371c3a9","Type":"ContainerStarted","Data":"3d4879cd6a08d59f09c431ff9d130698f2c379046dd33e0a8815fe3f440ba030"}
Sep 14 13:04:58.052634 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:58.052587    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq" podStartSLOduration=1.6045247969999998 podStartE2EDuration="6.052569777s" totalImagesPullingTime="4.44804498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:04:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:04:58.052308759 +0000 UTC m=+391.343642577" watchObservedRunningTime="2026-09-14 13:04:58.052569777 +0000 UTC m=+391.343903598"
Sep 14 13:04:59.044068 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:59.044028    2581 generic.go:356] "Generic (PLEG): container finished" podID="527f4585-b998-46ec-a189-ff50c371c3a9" containerID="3d4879cd6a08d59f09c431ff9d130698f2c379046dd33e0a8815fe3f440ba030" exitCode=0
Sep 14 13:04:59.044436 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:04:59.044064    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg" event={"ID":"527f4585-b998-46ec-a189-ff50c371c3a9","Type":"ContainerDied","Data":"3d4879cd6a08d59f09c431ff9d130698f2c379046dd33e0a8815fe3f440ba030"}
Sep 14 13:05:00.050722 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:00.050687    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg" event={"ID":"527f4585-b998-46ec-a189-ff50c371c3a9","Type":"ContainerStarted","Data":"ac408449efc193faf3b7867afed569540b08c43f7ed372674d1b4e7d758e9d66"}
Sep 14 13:05:00.051132 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:00.050835    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:05:00.070681 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:00.070619    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg" podStartSLOduration=1.618004403 podStartE2EDuration="7.070601482s" totalImagesPullingTime="4.396235959s" totalInitContainerRuntime="1.05636112s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:04:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:05:00.069601096 +0000 UTC m=+393.360934917" watchObservedRunningTime="2026-09-14 13:05:00.070601482 +0000 UTC m=+393.361935299"
Sep 14 13:05:09.047028 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:09.046997    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6d5f86f746-p8fmq"
Sep 14 13:05:11.057636 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:11.057606    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-697cf97c89-mwdgg"
Sep 14 13:05:12.577259 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:12.577207    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:12.704214 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:12.704177    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:23.375801 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.375765    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:23.433985 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.433946    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk"]
Sep 14 13:05:23.437206 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.437177    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk"
Sep 14 13:05:23.441675 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.441652    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 14 13:05:23.443052 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.442763    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 14 13:05:23.443052 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.442803    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7p42f\""
Sep 14 13:05:23.443052 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.442832    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 14 13:05:23.443052 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.442777    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 14 13:05:23.456677 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.456643    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk"]
Sep 14 13:05:23.525769 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.525727    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/662db2d6-5df3-499a-9052-8be6ec86e6e3-tmp\") pod \"kube-auth-proxy-7f6f8d7966-tzrvk\" (UID: \"662db2d6-5df3-499a-9052-8be6ec86e6e3\") " pod="openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk"
Sep 14 13:05:23.525952 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.525793    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/662db2d6-5df3-499a-9052-8be6ec86e6e3-tls-certs\") pod \"kube-auth-proxy-7f6f8d7966-tzrvk\" (UID: \"662db2d6-5df3-499a-9052-8be6ec86e6e3\") " pod="openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk"
Sep 14 13:05:23.525952 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.525876    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glqvb\" (UniqueName: \"kubernetes.io/projected/662db2d6-5df3-499a-9052-8be6ec86e6e3-kube-api-access-glqvb\") pod \"kube-auth-proxy-7f6f8d7966-tzrvk\" (UID: \"662db2d6-5df3-499a-9052-8be6ec86e6e3\") " pod="openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk"
Sep 14 13:05:23.529400 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.529363    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:23.627011 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.626923    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/662db2d6-5df3-499a-9052-8be6ec86e6e3-tmp\") pod \"kube-auth-proxy-7f6f8d7966-tzrvk\" (UID: \"662db2d6-5df3-499a-9052-8be6ec86e6e3\") " pod="openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk"
Sep 14 13:05:23.627148 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.627017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/662db2d6-5df3-499a-9052-8be6ec86e6e3-tls-certs\") pod \"kube-auth-proxy-7f6f8d7966-tzrvk\" (UID: \"662db2d6-5df3-499a-9052-8be6ec86e6e3\") " pod="openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk"
Sep 14 13:05:23.627148 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.627064    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glqvb\" (UniqueName: \"kubernetes.io/projected/662db2d6-5df3-499a-9052-8be6ec86e6e3-kube-api-access-glqvb\") pod \"kube-auth-proxy-7f6f8d7966-tzrvk\" (UID: \"662db2d6-5df3-499a-9052-8be6ec86e6e3\") " pod="openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk"
Sep 14 13:05:23.629121 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.629087    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/662db2d6-5df3-499a-9052-8be6ec86e6e3-tmp\") pod \"kube-auth-proxy-7f6f8d7966-tzrvk\" (UID: \"662db2d6-5df3-499a-9052-8be6ec86e6e3\") " pod="openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk"
Sep 14 13:05:23.629384 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.629365    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/662db2d6-5df3-499a-9052-8be6ec86e6e3-tls-certs\") pod \"kube-auth-proxy-7f6f8d7966-tzrvk\" (UID: \"662db2d6-5df3-499a-9052-8be6ec86e6e3\") " pod="openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk"
Sep 14 13:05:23.635139 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.635114    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glqvb\" (UniqueName: \"kubernetes.io/projected/662db2d6-5df3-499a-9052-8be6ec86e6e3-kube-api-access-glqvb\") pod \"kube-auth-proxy-7f6f8d7966-tzrvk\" (UID: \"662db2d6-5df3-499a-9052-8be6ec86e6e3\") " pod="openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk"
Sep 14 13:05:23.748752 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:23.748705    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk"
Sep 14 13:05:24.033048 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:24.033024    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk"]
Sep 14 13:05:24.035321 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:05:24.035290    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod662db2d6_5df3_499a_9052_8be6ec86e6e3.slice/crio-414b9c947b1666c1efe581410224096ce43ef7943068f5f8614b2d62cded964a WatchSource:0}: Error finding container 414b9c947b1666c1efe581410224096ce43ef7943068f5f8614b2d62cded964a: Status 404 returned error can't find the container with id 414b9c947b1666c1efe581410224096ce43ef7943068f5f8614b2d62cded964a
Sep 14 13:05:24.154867 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:24.154829    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk" event={"ID":"662db2d6-5df3-499a-9052-8be6ec86e6e3","Type":"ContainerStarted","Data":"414b9c947b1666c1efe581410224096ce43ef7943068f5f8614b2d62cded964a"}
Sep 14 13:05:24.604557 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:24.604504    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:24.727873 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:24.727781    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:25.652187 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:25.652148    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:27.170828 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:27.170793    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk" event={"ID":"662db2d6-5df3-499a-9052-8be6ec86e6e3","Type":"ContainerStarted","Data":"f8e99ece5a6aea7fc4a682242e25ce915500fdd05f2dfa1e6b804ae81cc3b7af"}
Sep 14 13:05:27.189856 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:27.189774    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7f6f8d7966-tzrvk" podStartSLOduration=1.6900100789999999 podStartE2EDuration="4.189754171s" totalImagesPullingTime="2.499744092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:05:27.188511302 +0000 UTC m=+420.479845132" watchObservedRunningTime="2026-09-14 13:05:27.189754171 +0000 UTC m=+420.481087988"
Sep 14 13:05:29.874339 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:29.874301    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:29.973268 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:29.973232    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:30.452193 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:30.452159    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:30.602285 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:30.602240    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:30.705255 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:30.705177    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:33.957987 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:33.957946    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:34.052164 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:34.052130    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:34.153712 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:34.153675    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:34.558883 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:34.558842    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:34.661566 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:34.661517    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:34.812115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:34.812035    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:43.114105 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:43.114047    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:43.212155 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:43.212098    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:43.325606 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:43.325567    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:43.723643 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:43.723591    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:44.409124 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:44.409091    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:44.536583 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:05:44.536519    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:23.192772 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:23.192710    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:35.009330 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:35.009296    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:35.335382 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:35.335298    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:46.204104 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:46.204067    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:46.947599 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:46.947562    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:47.732998 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:47.732955    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:50.004254 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:50.004222    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:50.103521 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:50.103487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:51.542218 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:51.542182    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:52.392263 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:52.392227    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:53.546231 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:53.546188    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:53.810025 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:53.809941    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:55.342272 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:55.342241    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:56.157235 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:56.157199    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:56.257341 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:56.257307    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:58.229120 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:58.229089    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:58.567601 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:06:58.567475    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:00.939812 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:00.939777    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:00.996839 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:00.996807    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:01.348278 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:01.348195    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:01.413689 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:01.413659    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:01.746350 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:01.746314    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:01.976254 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:01.976218    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:07.401178 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:07.401141    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:07.503139 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:07.503107    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:10.300087 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.300050    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"]
Sep 14 13:07:10.304215 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.304186    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"
Sep 14 13:07:10.307098 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.307069    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 14 13:07:10.307264 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.307080    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fbs72\""
Sep 14 13:07:10.307447 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.307427    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 14 13:07:10.313743 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.313706    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"]
Sep 14 13:07:10.460058 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.460009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj2kr\" (UniqueName: \"kubernetes.io/projected/45772d7c-5790-4642-940a-335038b9560d-kube-api-access-pj2kr\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w6hzm\" (UID: \"45772d7c-5790-4642-940a-335038b9560d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"
Sep 14 13:07:10.460058 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.460063    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/45772d7c-5790-4642-940a-335038b9560d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w6hzm\" (UID: \"45772d7c-5790-4642-940a-335038b9560d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"
Sep 14 13:07:10.561045 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.560945    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj2kr\" (UniqueName: \"kubernetes.io/projected/45772d7c-5790-4642-940a-335038b9560d-kube-api-access-pj2kr\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w6hzm\" (UID: \"45772d7c-5790-4642-940a-335038b9560d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"
Sep 14 13:07:10.561045 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.561006    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/45772d7c-5790-4642-940a-335038b9560d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w6hzm\" (UID: \"45772d7c-5790-4642-940a-335038b9560d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"
Sep 14 13:07:10.561357 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.561341    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/45772d7c-5790-4642-940a-335038b9560d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w6hzm\" (UID: \"45772d7c-5790-4642-940a-335038b9560d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"
Sep 14 13:07:10.571771 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.571737    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj2kr\" (UniqueName: \"kubernetes.io/projected/45772d7c-5790-4642-940a-335038b9560d-kube-api-access-pj2kr\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w6hzm\" (UID: \"45772d7c-5790-4642-940a-335038b9560d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"
Sep 14 13:07:10.617915 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.617875    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"
Sep 14 13:07:10.690433 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.690393    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:10.894245 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:10.894009    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"]
Sep 14 13:07:10.897196 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:07:10.897165    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45772d7c_5790_4642_940a_335038b9560d.slice/crio-baf3b781798ac244003bd4f97c9546f0acc075f9cb04817a46423337cfbb1f22 WatchSource:0}: Error finding container baf3b781798ac244003bd4f97c9546f0acc075f9cb04817a46423337cfbb1f22: Status 404 returned error can't find the container with id baf3b781798ac244003bd4f97c9546f0acc075f9cb04817a46423337cfbb1f22
Sep 14 13:07:11.631302 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:11.631266    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm" event={"ID":"45772d7c-5790-4642-940a-335038b9560d","Type":"ContainerStarted","Data":"baf3b781798ac244003bd4f97c9546f0acc075f9cb04817a46423337cfbb1f22"}
Sep 14 13:07:12.186457 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:12.186383    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:12.271760 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:12.271724    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:15.661306 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:15.661258    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm" event={"ID":"45772d7c-5790-4642-940a-335038b9560d","Type":"ContainerStarted","Data":"95998d2dac2e70525ea06b732475d3428a684578e6144d2cc7cf26809fb28ea3"}
Sep 14 13:07:15.661704 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:15.661483    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"
Sep 14 13:07:15.681263 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:15.681214    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm" podStartSLOduration=1.882816814 podStartE2EDuration="5.681197747s" totalImagesPullingTime="3.798380933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:07:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:07:15.67993933 +0000 UTC m=+528.971273138" watchObservedRunningTime="2026-09-14 13:07:15.681197747 +0000 UTC m=+528.972531563"
Sep 14 13:07:16.961212 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:16.961177    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:17.061137 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:17.061099    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:22.819896 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:22.819858    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:22.883081 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:22.883043    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:23.686076 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:23.686045    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:23.784719 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:23.784687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:26.668677 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:26.668650    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"
Sep 14 13:07:29.210281 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:29.210241    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:29.329100 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:29.329068    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:29.537685 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:29.537604    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:33.334181 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:33.334148    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:33.969684 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:33.969645    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:34.081941 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:34.081909    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:39.301423 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:39.301392    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:39.357738 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:39.357710    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:39.888845 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:39.888806    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:39.946473 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:39.946446    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:43.080818 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:43.080785    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:43.172858 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:43.172820    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:47.473204 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:47.473166    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:47.576361 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:47.576327    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:47.649062 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:47.649030    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:47.863073 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:47.863042    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:07:47.867949 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:47.867928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"
Sep 14 13:07:47.870335 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:47.870309    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 14 13:07:47.870591 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:47.870575    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bznfl\""
Sep 14 13:07:47.875620 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:47.875598    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:07:47.904661 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:47.904629    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:07:47.988815 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:47.988782    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46dc4cee-e2f4-4854-b6c9-264795d5c70f-config-file\") pod \"limitador-limitador-84994f7ffc-pjqrn\" (UID: \"46dc4cee-e2f4-4854-b6c9-264795d5c70f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"
Sep 14 13:07:47.988991 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:47.988830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkv95\" (UniqueName: \"kubernetes.io/projected/46dc4cee-e2f4-4854-b6c9-264795d5c70f-kube-api-access-lkv95\") pod \"limitador-limitador-84994f7ffc-pjqrn\" (UID: \"46dc4cee-e2f4-4854-b6c9-264795d5c70f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"
Sep 14 13:07:48.090097 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:48.090053    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46dc4cee-e2f4-4854-b6c9-264795d5c70f-config-file\") pod \"limitador-limitador-84994f7ffc-pjqrn\" (UID: \"46dc4cee-e2f4-4854-b6c9-264795d5c70f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"
Sep 14 13:07:48.090252 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:48.090112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkv95\" (UniqueName: \"kubernetes.io/projected/46dc4cee-e2f4-4854-b6c9-264795d5c70f-kube-api-access-lkv95\") pod \"limitador-limitador-84994f7ffc-pjqrn\" (UID: \"46dc4cee-e2f4-4854-b6c9-264795d5c70f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"
Sep 14 13:07:48.090724 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:48.090706    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/46dc4cee-e2f4-4854-b6c9-264795d5c70f-config-file\") pod \"limitador-limitador-84994f7ffc-pjqrn\" (UID: \"46dc4cee-e2f4-4854-b6c9-264795d5c70f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"
Sep 14 13:07:48.098620 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:48.098593    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkv95\" (UniqueName: \"kubernetes.io/projected/46dc4cee-e2f4-4854-b6c9-264795d5c70f-kube-api-access-lkv95\") pod \"limitador-limitador-84994f7ffc-pjqrn\" (UID: \"46dc4cee-e2f4-4854-b6c9-264795d5c70f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"
Sep 14 13:07:48.180391 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:48.180302    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"
Sep 14 13:07:48.314978 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:48.314935    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:48.420193 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:48.420140    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:48.450417 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:48.450392    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:07:48.455049 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:07:48.454805    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46dc4cee_e2f4_4854_b6c9_264795d5c70f.slice/crio-e76e8022b3c0506c0af9c351b0d86c12b359cd8a607b9cb6ed49437ecf885a38 WatchSource:0}: Error finding container e76e8022b3c0506c0af9c351b0d86c12b359cd8a607b9cb6ed49437ecf885a38: Status 404 returned error can't find the container with id e76e8022b3c0506c0af9c351b0d86c12b359cd8a607b9cb6ed49437ecf885a38
Sep 14 13:07:48.812035 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:48.811951    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-pjqrn" event={"ID":"46dc4cee-e2f4-4854-b6c9-264795d5c70f","Type":"ContainerStarted","Data":"e76e8022b3c0506c0af9c351b0d86c12b359cd8a607b9cb6ed49437ecf885a38"}
Sep 14 13:07:49.818702 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:49.818606    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-pjqrn" event={"ID":"46dc4cee-e2f4-4854-b6c9-264795d5c70f","Type":"ContainerStarted","Data":"4b22f20831052e09cf239877b5e48e9b16c5485f91b8d5399ccf4f5e893a5505"}
Sep 14 13:07:49.819098 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:49.818725    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"
Sep 14 13:07:49.834024 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:49.833959    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-pjqrn" podStartSLOduration=1.902130449 podStartE2EDuration="2.833935047s" totalImagesPullingTime="931.804598ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:07:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:07:49.833603482 +0000 UTC m=+563.124937301" watchObservedRunningTime="2026-09-14 13:07:49.833935047 +0000 UTC m=+563.125268864"
Sep 14 13:07:53.177525 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:07:53.177487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:00.824367 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:00.824331    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"
Sep 14 13:08:03.693109 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:03.693072    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:03.794091 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:03.794051    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:04.146251 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:04.146222    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:09.874004 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:09.873952    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:09.976133 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:09.976094    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:13.865751 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:13.865709    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:14.130204 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:14.130113    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:21.522220 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:21.522167    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:21.593039 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:21.593011    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:21.650835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:21.650797    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:21.794338 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:21.794246    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:21.857303 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:21.857272    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:24.211907 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:24.211879    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:24.272366 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:24.272330    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:24.939873 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:24.939840    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:25.031404 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:25.031373    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:27.150275 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:27.150245    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:08:27.150723 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:27.150245    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:08:55.487556 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:55.487464    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:56.710520 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:56.710482    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:56.857909 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:56.857875    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:56.958884 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:56.958831    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:57.060210 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:57.060136    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:57.257414 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:57.257386    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:57.638126 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:57.638095    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:57.738500 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:57.738467    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:08:57.882733 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:08:57.882702    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:09:07.981119 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:09:07.981085    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:09:08.081343 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:09:08.081311    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:09:20.822937 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:09:20.822900    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:09:21.278617 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:09:21.278574    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:09:29.848885 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:09:29.848844    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:09:30.458391 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:09:30.458350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:09:40.434541 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:09:40.434492    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:09:41.364599 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:09:41.364564    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:10:00.709195 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.709162    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"]
Sep 14 13:10:00.712572 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.712554    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.716315 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.716281    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 14 13:10:00.716463 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.716281    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 14 13:10:00.716463 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.716281    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4m98s\""
Sep 14 13:10:00.716463 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.716394    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 14 13:10:00.724385 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.724362    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"]
Sep 14 13:10:00.757130 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.757102    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:10:00.769015 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.768991    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-264sf\" (UniqueName: \"kubernetes.io/projected/7baae410-2e01-4d0f-aa09-e85fdc179263-kube-api-access-264sf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.769110 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.769027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7baae410-2e01-4d0f-aa09-e85fdc179263-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.769110 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.769045    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7baae410-2e01-4d0f-aa09-e85fdc179263-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.769222 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.769131    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7baae410-2e01-4d0f-aa09-e85fdc179263-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.769222 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.769156    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7baae410-2e01-4d0f-aa09-e85fdc179263-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.769222 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.769174    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7baae410-2e01-4d0f-aa09-e85fdc179263-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.870412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.870368    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-264sf\" (UniqueName: \"kubernetes.io/projected/7baae410-2e01-4d0f-aa09-e85fdc179263-kube-api-access-264sf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.870637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.870421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7baae410-2e01-4d0f-aa09-e85fdc179263-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.870637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.870552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7baae410-2e01-4d0f-aa09-e85fdc179263-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.870637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.870624    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7baae410-2e01-4d0f-aa09-e85fdc179263-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.870816 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.870649    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7baae410-2e01-4d0f-aa09-e85fdc179263-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.870816 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.870674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7baae410-2e01-4d0f-aa09-e85fdc179263-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.870816 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.870794    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7baae410-2e01-4d0f-aa09-e85fdc179263-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.870956 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.870935    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7baae410-2e01-4d0f-aa09-e85fdc179263-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.871042 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.871023    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7baae410-2e01-4d0f-aa09-e85fdc179263-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.872912 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.872885    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7baae410-2e01-4d0f-aa09-e85fdc179263-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.873091 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.873075    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7baae410-2e01-4d0f-aa09-e85fdc179263-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:00.878447 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:00.878428    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-264sf\" (UniqueName: \"kubernetes.io/projected/7baae410-2e01-4d0f-aa09-e85fdc179263-kube-api-access-264sf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mj64s\" (UID: \"7baae410-2e01-4d0f-aa09-e85fdc179263\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:01.025514 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:01.025423    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:01.298582 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:01.298373    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"]
Sep 14 13:10:01.301406 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:10:01.301381    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7baae410_2e01_4d0f_aa09_e85fdc179263.slice/crio-c0feed9bd1d6a700b1c5adc68336f244e2104c56d34375aeb17da1c1b27505d1 WatchSource:0}: Error finding container c0feed9bd1d6a700b1c5adc68336f244e2104c56d34375aeb17da1c1b27505d1: Status 404 returned error can't find the container with id c0feed9bd1d6a700b1c5adc68336f244e2104c56d34375aeb17da1c1b27505d1
Sep 14 13:10:01.302774 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:01.302754    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 13:10:01.420204 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:01.420168    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s" event={"ID":"7baae410-2e01-4d0f-aa09-e85fdc179263","Type":"ContainerStarted","Data":"c0feed9bd1d6a700b1c5adc68336f244e2104c56d34375aeb17da1c1b27505d1"}
Sep 14 13:10:01.961390 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:01.961346    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:10:03.432329 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:03.432283    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s" event={"ID":"7baae410-2e01-4d0f-aa09-e85fdc179263","Type":"ContainerStarted","Data":"a23647a5c570b68b30e58cccbf518af8c5ad20885a811c705093a174f8654013"}
Sep 14 13:10:03.455918 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:03.455861    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s" podStartSLOduration=1.6673109350000002 podStartE2EDuration="3.455844836s" totalImagesPullingTime="1.788533901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:10:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:10:03.455049289 +0000 UTC m=+696.746383105" watchObservedRunningTime="2026-09-14 13:10:03.455844836 +0000 UTC m=+696.747178652"
Sep 14 13:10:10.536353 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:10.536319    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:10:11.025611 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:11.025571    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:11.025801 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:11.025638    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:11.037879 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:11.037853    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:11.479184 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:11.479154    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mj64s"
Sep 14 13:10:12.148460 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:12.148421    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:10:27.855856 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:27.855819    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:10:29.563301 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:29.563271    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:10:31.600267 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.600231    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"]
Sep 14 13:10:31.603890 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.603860    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.606698 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.606664    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 14 13:10:31.615719 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.615694    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"]
Sep 14 13:10:31.636725 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.636697    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:10:31.755606 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.755572    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4708d8c6-a33e-459f-b87c-616d1525de82-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.755772 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.755623    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlxrq\" (UniqueName: \"kubernetes.io/projected/4708d8c6-a33e-459f-b87c-616d1525de82-kube-api-access-hlxrq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.755772 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.755662    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4708d8c6-a33e-459f-b87c-616d1525de82-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.755772 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.755694    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4708d8c6-a33e-459f-b87c-616d1525de82-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.755881 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.755776    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4708d8c6-a33e-459f-b87c-616d1525de82-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.755881 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.755803    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4708d8c6-a33e-459f-b87c-616d1525de82-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.856856 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.856828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4708d8c6-a33e-459f-b87c-616d1525de82-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.857071 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.856869    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4708d8c6-a33e-459f-b87c-616d1525de82-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.857071 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.856896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4708d8c6-a33e-459f-b87c-616d1525de82-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.857071 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.856923    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlxrq\" (UniqueName: \"kubernetes.io/projected/4708d8c6-a33e-459f-b87c-616d1525de82-kube-api-access-hlxrq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.857071 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.856954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4708d8c6-a33e-459f-b87c-616d1525de82-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.857071 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.857026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4708d8c6-a33e-459f-b87c-616d1525de82-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.857362 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.857343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4708d8c6-a33e-459f-b87c-616d1525de82-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.857448 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.857420    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4708d8c6-a33e-459f-b87c-616d1525de82-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.857504 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.857457    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4708d8c6-a33e-459f-b87c-616d1525de82-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.859274 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.859255    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4708d8c6-a33e-459f-b87c-616d1525de82-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.859430 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.859413    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4708d8c6-a33e-459f-b87c-616d1525de82-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.865308 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.865289    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlxrq\" (UniqueName: \"kubernetes.io/projected/4708d8c6-a33e-459f-b87c-616d1525de82-kube-api-access-hlxrq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl\" (UID: \"4708d8c6-a33e-459f-b87c-616d1525de82\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:31.913873 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:31.913834    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:32.186578 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:32.186462    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"]
Sep 14 13:10:32.189552 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:10:32.189503    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4708d8c6_a33e_459f_b87c_616d1525de82.slice/crio-8aa542041e74cd8db8f03fa506aa8c12e2d5d1651961d364bbe9218371843f41 WatchSource:0}: Error finding container 8aa542041e74cd8db8f03fa506aa8c12e2d5d1651961d364bbe9218371843f41: Status 404 returned error can't find the container with id 8aa542041e74cd8db8f03fa506aa8c12e2d5d1651961d364bbe9218371843f41
Sep 14 13:10:32.565723 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:32.565690    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl" event={"ID":"4708d8c6-a33e-459f-b87c-616d1525de82","Type":"ContainerStarted","Data":"8aa542041e74cd8db8f03fa506aa8c12e2d5d1651961d364bbe9218371843f41"}
Sep 14 13:10:33.571908 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:33.571869    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl" event={"ID":"4708d8c6-a33e-459f-b87c-616d1525de82","Type":"ContainerStarted","Data":"3a1f431448cb4d32962f8c19a8fb02539ad4d6185e63178349bda298554a2cd0"}
Sep 14 13:10:33.590873 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:33.590815    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl" podStartSLOduration=2.234535847 podStartE2EDuration="2.59079839s" totalImagesPullingTime="356.262543ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:10:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:10:33.589800537 +0000 UTC m=+726.881134356" watchObservedRunningTime="2026-09-14 13:10:33.59079839 +0000 UTC m=+726.882132206"
Sep 14 13:10:37.761815 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:37.761778    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:10:41.913978 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:41.913940    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:41.914409 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:41.914026    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:41.927109 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:41.927078    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:42.633180 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:42.633149    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl"
Sep 14 13:10:44.995025 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:10:44.994990    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:01.187690 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:01.187606    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:01.287053 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:01.287015    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:02.278243 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:02.278176    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:02.378891 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:02.378843    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:17.009850 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:17.009815    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:17.109703 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:17.109666    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:17.217982 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:17.217938    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:17.368077 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:17.368042    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:17.458551 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:17.458494    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:17.535288 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:17.535250    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:17.807021 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:17.806926    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:18.003761 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:18.003715    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:18.378159 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:18.378121    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:18.477939 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:18.477908    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:18.578118 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:18.578081    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:18.834168 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:18.834069    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:19.032898 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:19.032861    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7431-f6537a"]
Sep 14 13:12:19.036349 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:19.036329    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7431-f6537a"
Sep 14 13:12:19.039103 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:19.039079    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vtxmg\""
Sep 14 13:12:19.043204 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:19.043178    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7431-f6537a"]
Sep 14 13:12:19.124164 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:19.124128    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxnls\" (UniqueName: \"kubernetes.io/projected/2b2bc118-676d-49ef-a642-6fad72bbf0cd-kube-api-access-qxnls\") pod \"test-curl-7431-f6537a\" (UID: \"2b2bc118-676d-49ef-a642-6fad72bbf0cd\") " pod="opendatahub/test-curl-7431-f6537a"
Sep 14 13:12:19.225093 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:19.225051    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxnls\" (UniqueName: \"kubernetes.io/projected/2b2bc118-676d-49ef-a642-6fad72bbf0cd-kube-api-access-qxnls\") pod \"test-curl-7431-f6537a\" (UID: \"2b2bc118-676d-49ef-a642-6fad72bbf0cd\") " pod="opendatahub/test-curl-7431-f6537a"
Sep 14 13:12:19.233868 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:19.233835    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxnls\" (UniqueName: \"kubernetes.io/projected/2b2bc118-676d-49ef-a642-6fad72bbf0cd-kube-api-access-qxnls\") pod \"test-curl-7431-f6537a\" (UID: \"2b2bc118-676d-49ef-a642-6fad72bbf0cd\") " pod="opendatahub/test-curl-7431-f6537a"
Sep 14 13:12:19.347682 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:19.347641    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7431-f6537a"
Sep 14 13:12:19.620076 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:19.620040    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7431-f6537a"]
Sep 14 13:12:19.623788 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:12:19.623753    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b2bc118_676d_49ef_a642_6fad72bbf0cd.slice/crio-784465ab360255a1bfd2816aa955107cf28816f4092440c7a04b7b93eedcb09b WatchSource:0}: Error finding container 784465ab360255a1bfd2816aa955107cf28816f4092440c7a04b7b93eedcb09b: Status 404 returned error can't find the container with id 784465ab360255a1bfd2816aa955107cf28816f4092440c7a04b7b93eedcb09b
Sep 14 13:12:20.018420 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:20.018384    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:20.067932 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:20.067897    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7431-f6537a" event={"ID":"2b2bc118-676d-49ef-a642-6fad72bbf0cd","Type":"ContainerStarted","Data":"784465ab360255a1bfd2816aa955107cf28816f4092440c7a04b7b93eedcb09b"}
Sep 14 13:12:20.114858 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:20.114822    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:20.262128 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:20.262087    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:12:20.696937 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:20.696901    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:20.851520 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:20.851479    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:20.974426 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:20.974312    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:21.750164 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:21.750085    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:21.848711 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:21.848680    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:22.078378 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.078281    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7431-f6537a" event={"ID":"2b2bc118-676d-49ef-a642-6fad72bbf0cd","Type":"ContainerStarted","Data":"2223b1292eb7b52cdb5474b33a00f0bf3cc181b2392ec1050169f1b70ef37c9f"}
Sep 14 13:12:22.096145 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.096096    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7431-f6537a" podStartSLOduration=1.077307871 podStartE2EDuration="3.09607871s" totalImagesPullingTime="2.018770839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:22.094584973 +0000 UTC m=+835.385918790" watchObservedRunningTime="2026-09-14 13:12:22.09607871 +0000 UTC m=+835.387412527"
Sep 14 13:12:22.121040 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.121002    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:22.226244 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.226210    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:22.330148 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.330026    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:22.678475 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.678433    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7431-f6537a"]
Sep 14 13:12:22.686411 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.686377    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7431-f6537a"]
Sep 14 13:12:22.767867 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.767836    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:22.819549 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.819497    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7431-327b98"]
Sep 14 13:12:22.819997 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.819977    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b2bc118-676d-49ef-a642-6fad72bbf0cd" containerName="test-curl-7431-f6537a"
Sep 14 13:12:22.820088 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.820000    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b2bc118-676d-49ef-a642-6fad72bbf0cd" containerName="test-curl-7431-f6537a"
Sep 14 13:12:22.820146 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.820094    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b2bc118-676d-49ef-a642-6fad72bbf0cd" containerName="test-curl-7431-f6537a"
Sep 14 13:12:22.823286 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.823261    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7431-327b98"
Sep 14 13:12:22.827144 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.827106    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7431-327b98"]
Sep 14 13:12:22.868888 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.868845    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:22.961417 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:22.961323    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8mjc\" (UniqueName: \"kubernetes.io/projected/518a90b4-f2a8-4a39-944e-50a41f5080dc-kube-api-access-k8mjc\") pod \"test-curl-7431-327b98\" (UID: \"518a90b4-f2a8-4a39-944e-50a41f5080dc\") " pod="opendatahub/test-curl-7431-327b98"
Sep 14 13:12:23.062909 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:23.062864    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8mjc\" (UniqueName: \"kubernetes.io/projected/518a90b4-f2a8-4a39-944e-50a41f5080dc-kube-api-access-k8mjc\") pod \"test-curl-7431-327b98\" (UID: \"518a90b4-f2a8-4a39-944e-50a41f5080dc\") " pod="opendatahub/test-curl-7431-327b98"
Sep 14 13:12:23.074366 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:23.074335    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8mjc\" (UniqueName: \"kubernetes.io/projected/518a90b4-f2a8-4a39-944e-50a41f5080dc-kube-api-access-k8mjc\") pod \"test-curl-7431-327b98\" (UID: \"518a90b4-f2a8-4a39-944e-50a41f5080dc\") " pod="opendatahub/test-curl-7431-327b98"
Sep 14 13:12:23.085128 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:12:23.085095    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7431-f6537a" container="test-curl-7431-f6537a"
Sep 14 13:12:23.135835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:23.135802    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7431-327b98"
Sep 14 13:12:23.209091 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:23.208960    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:23.304324 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:23.304236    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:23.397677 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:23.397644    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:23.417443 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:23.417422    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7431-327b98"]
Sep 14 13:12:23.420429 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:12:23.420398    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod518a90b4_f2a8_4a39_944e_50a41f5080dc.slice/crio-8594932d547a51a813867c76cc1aa686c3cacbb19f00943a01f77331d924257e WatchSource:0}: Error finding container 8594932d547a51a813867c76cc1aa686c3cacbb19f00943a01f77331d924257e: Status 404 returned error can't find the container with id 8594932d547a51a813867c76cc1aa686c3cacbb19f00943a01f77331d924257e
Sep 14 13:12:23.666389 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:23.666337    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:23.779542 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:23.779505    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:23.929324 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:23.929245    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:24.091325 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.091287    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7431-327b98" event={"ID":"518a90b4-f2a8-4a39-944e-50a41f5080dc","Type":"ContainerStarted","Data":"0464e75075459bc33ea26fd49c57937d2f842fb9b346c742cc91249464bc3d28"}
Sep 14 13:12:24.091522 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.091335    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7431-327b98" event={"ID":"518a90b4-f2a8-4a39-944e-50a41f5080dc","Type":"ContainerStarted","Data":"8594932d547a51a813867c76cc1aa686c3cacbb19f00943a01f77331d924257e"}
Sep 14 13:12:24.091522 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.091438    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7431-f6537a" podUID="2b2bc118-676d-49ef-a642-6fad72bbf0cd" containerName="test-curl-7431-f6537a" containerID="cri-o://2223b1292eb7b52cdb5474b33a00f0bf3cc181b2392ec1050169f1b70ef37c9f" gracePeriod=2
Sep 14 13:12:24.107816 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.107767    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7431-327b98" podStartSLOduration=2.107746667 podStartE2EDuration="2.107746667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:24.105735685 +0000 UTC m=+837.397069504" watchObservedRunningTime="2026-09-14 13:12:24.107746667 +0000 UTC m=+837.399080485"
Sep 14 13:12:24.144924 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.144889    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:24.245435 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.245346    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:24.246302 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.246279    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"]
Sep 14 13:12:24.249771 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.249754    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"
Sep 14 13:12:24.252127 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.252102    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-975227\""
Sep 14 13:12:24.252253 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.252137    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-975227-dockercfg-nw6k6\""
Sep 14 13:12:24.259459 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.259429    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"]
Sep 14 13:12:24.274896 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.274858    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"]
Sep 14 13:12:24.278314 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.278293    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"
Sep 14 13:12:24.288392 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.288359    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"]
Sep 14 13:12:24.371328 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.371291    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jtmk\" (UniqueName: \"kubernetes.io/projected/20398542-b3f0-4df2-8485-d3fe43b8aaba-kube-api-access-4jtmk\") pod \"payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q\" (UID: \"20398542-b3f0-4df2-8485-d3fe43b8aaba\") " pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"
Sep 14 13:12:24.371525 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.371354    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20398542-b3f0-4df2-8485-d3fe43b8aaba-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q\" (UID: \"20398542-b3f0-4df2-8485-d3fe43b8aaba\") " pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"
Sep 14 13:12:24.371525 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.371415    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d56be6b-d25a-4727-b0e6-5b6baef100da-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54\" (UID: \"8d56be6b-d25a-4727-b0e6-5b6baef100da\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"
Sep 14 13:12:24.371525 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.371438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgxnz\" (UniqueName: \"kubernetes.io/projected/8d56be6b-d25a-4727-b0e6-5b6baef100da-kube-api-access-jgxnz\") pod \"payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54\" (UID: \"8d56be6b-d25a-4727-b0e6-5b6baef100da\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"
Sep 14 13:12:24.472633 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.472606    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20398542-b3f0-4df2-8485-d3fe43b8aaba-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q\" (UID: \"20398542-b3f0-4df2-8485-d3fe43b8aaba\") " pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"
Sep 14 13:12:24.472739 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.472648    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d56be6b-d25a-4727-b0e6-5b6baef100da-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54\" (UID: \"8d56be6b-d25a-4727-b0e6-5b6baef100da\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"
Sep 14 13:12:24.472739 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.472669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgxnz\" (UniqueName: \"kubernetes.io/projected/8d56be6b-d25a-4727-b0e6-5b6baef100da-kube-api-access-jgxnz\") pod \"payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54\" (UID: \"8d56be6b-d25a-4727-b0e6-5b6baef100da\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"
Sep 14 13:12:24.472739 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.472727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jtmk\" (UniqueName: \"kubernetes.io/projected/20398542-b3f0-4df2-8485-d3fe43b8aaba-kube-api-access-4jtmk\") pod \"payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q\" (UID: \"20398542-b3f0-4df2-8485-d3fe43b8aaba\") " pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"
Sep 14 13:12:24.473453 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.473429    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20398542-b3f0-4df2-8485-d3fe43b8aaba-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q\" (UID: \"20398542-b3f0-4df2-8485-d3fe43b8aaba\") " pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"
Sep 14 13:12:24.473571 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.473435    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d56be6b-d25a-4727-b0e6-5b6baef100da-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54\" (UID: \"8d56be6b-d25a-4727-b0e6-5b6baef100da\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"
Sep 14 13:12:24.481412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.481381    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgxnz\" (UniqueName: \"kubernetes.io/projected/8d56be6b-d25a-4727-b0e6-5b6baef100da-kube-api-access-jgxnz\") pod \"payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54\" (UID: \"8d56be6b-d25a-4727-b0e6-5b6baef100da\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"
Sep 14 13:12:24.481481 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.481438    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jtmk\" (UniqueName: \"kubernetes.io/projected/20398542-b3f0-4df2-8485-d3fe43b8aaba-kube-api-access-4jtmk\") pod \"payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q\" (UID: \"20398542-b3f0-4df2-8485-d3fe43b8aaba\") " pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"
Sep 14 13:12:24.561162 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.561071    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"
Sep 14 13:12:24.590009 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.589978    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"
Sep 14 13:12:24.626950 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.625058    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:24.687040 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.686998    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7431-327b98"]
Sep 14 13:12:24.692978 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.692948    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7431-327b98"]
Sep 14 13:12:24.840667 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.840592    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"]
Sep 14 13:12:24.844114 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:12:24.844089    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d56be6b_d25a_4727_b0e6_5b6baef100da.slice/crio-f3352374ee0ea5ce2d1b2c64019967193b8c81fd4ac7514dd72128335dfd2e98 WatchSource:0}: Error finding container f3352374ee0ea5ce2d1b2c64019967193b8c81fd4ac7514dd72128335dfd2e98: Status 404 returned error can't find the container with id f3352374ee0ea5ce2d1b2c64019967193b8c81fd4ac7514dd72128335dfd2e98
Sep 14 13:12:24.868682 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:24.868658    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"]
Sep 14 13:12:24.871194 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:12:24.871168    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20398542_b3f0_4df2_8485_d3fe43b8aaba.slice/crio-b3eb96166abec0e5dd76e37fcec2d8973ed26c124e30238075e38cdcaea704f1 WatchSource:0}: Error finding container b3eb96166abec0e5dd76e37fcec2d8973ed26c124e30238075e38cdcaea704f1: Status 404 returned error can't find the container with id b3eb96166abec0e5dd76e37fcec2d8973ed26c124e30238075e38cdcaea704f1
Sep 14 13:12:25.098404 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:25.098297    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q" event={"ID":"20398542-b3f0-4df2-8485-d3fe43b8aaba","Type":"ContainerStarted","Data":"b3eb96166abec0e5dd76e37fcec2d8973ed26c124e30238075e38cdcaea704f1"}
Sep 14 13:12:25.099660 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:25.099629    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54" event={"ID":"8d56be6b-d25a-4727-b0e6-5b6baef100da","Type":"ContainerStarted","Data":"f3352374ee0ea5ce2d1b2c64019967193b8c81fd4ac7514dd72128335dfd2e98"}
Sep 14 13:12:25.099908 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:12:25.099893    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7431-327b98" container="test-curl-7431-327b98"
Sep 14 13:12:25.201286 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:25.201257    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:25.300778 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:25.300745    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:25.475702 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:25.475657    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:25.906693 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:25.906645    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:26.007052 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:26.007013    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:26.107746 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:26.107468    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7431-327b98" podUID="518a90b4-f2a8-4a39-944e-50a41f5080dc" containerName="test-curl-7431-327b98" containerID="cri-o://0464e75075459bc33ea26fd49c57937d2f842fb9b346c742cc91249464bc3d28" gracePeriod=2
Sep 14 13:12:26.320040 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:26.320011    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7431-f6537a"
Sep 14 13:12:26.390870 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:26.390801    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxnls\" (UniqueName: \"kubernetes.io/projected/2b2bc118-676d-49ef-a642-6fad72bbf0cd-kube-api-access-qxnls\") pod \"2b2bc118-676d-49ef-a642-6fad72bbf0cd\" (UID: \"2b2bc118-676d-49ef-a642-6fad72bbf0cd\") "
Sep 14 13:12:26.393055 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:26.393026    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b2bc118-676d-49ef-a642-6fad72bbf0cd-kube-api-access-qxnls" (OuterVolumeSpecName: "kube-api-access-qxnls") pod "2b2bc118-676d-49ef-a642-6fad72bbf0cd" (UID: "2b2bc118-676d-49ef-a642-6fad72bbf0cd"). InnerVolumeSpecName "kube-api-access-qxnls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:12:26.492645 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:26.492613    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxnls\" (UniqueName: \"kubernetes.io/projected/2b2bc118-676d-49ef-a642-6fad72bbf0cd-kube-api-access-qxnls\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:12:26.578455 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:26.578419    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:26.644423 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:26.644384    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:26.807223 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:26.807123    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:27.087376 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:27.087297    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:27.113709 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:27.113674    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q" event={"ID":"20398542-b3f0-4df2-8485-d3fe43b8aaba","Type":"ContainerStarted","Data":"322845275d7c60e3c1c9c4ace61fb1ea43cf0331b34664a27ae50efd71e8edfa"}
Sep 14 13:12:27.113901 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:27.113771    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"
Sep 14 13:12:27.115384 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:27.115353    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54" event={"ID":"8d56be6b-d25a-4727-b0e6-5b6baef100da","Type":"ContainerStarted","Data":"e190268ccbeae8e704088266faf63af1956da58730c29be0d35dd89f33e8a53f"}
Sep 14 13:12:27.115517 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:27.115413    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"
Sep 14 13:12:27.116792 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:27.116766    2581 generic.go:356] "Generic (PLEG): container finished" podID="2b2bc118-676d-49ef-a642-6fad72bbf0cd" containerID="2223b1292eb7b52cdb5474b33a00f0bf3cc181b2392ec1050169f1b70ef37c9f" exitCode=137
Sep 14 13:12:27.116885 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:27.116810    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7431-f6537a"
Sep 14 13:12:27.116885 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:27.116870    2581 scope.go:117] "RemoveContainer" containerID="2223b1292eb7b52cdb5474b33a00f0bf3cc181b2392ec1050169f1b70ef37c9f"
Sep 14 13:12:27.125961 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:27.125945    2581 scope.go:117] "RemoveContainer" containerID="2223b1292eb7b52cdb5474b33a00f0bf3cc181b2392ec1050169f1b70ef37c9f"
Sep 14 13:12:27.126256 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:12:27.126235    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2223b1292eb7b52cdb5474b33a00f0bf3cc181b2392ec1050169f1b70ef37c9f\": container with ID starting with 2223b1292eb7b52cdb5474b33a00f0bf3cc181b2392ec1050169f1b70ef37c9f not found: ID does not exist" containerID="2223b1292eb7b52cdb5474b33a00f0bf3cc181b2392ec1050169f1b70ef37c9f"
Sep 14 13:12:27.126302 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:27.126264    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2223b1292eb7b52cdb5474b33a00f0bf3cc181b2392ec1050169f1b70ef37c9f"} err="failed to get container status \"2223b1292eb7b52cdb5474b33a00f0bf3cc181b2392ec1050169f1b70ef37c9f\": rpc error: code = NotFound desc = could not find container \"2223b1292eb7b52cdb5474b33a00f0bf3cc181b2392ec1050169f1b70ef37c9f\": container with ID starting with 2223b1292eb7b52cdb5474b33a00f0bf3cc181b2392ec1050169f1b70ef37c9f not found: ID does not exist"
Sep 14 13:12:27.131644 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:27.131602    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q" podStartSLOduration=1.8316153979999998 podStartE2EDuration="3.131589557s" totalImagesPullingTime="1.299974159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:27.130679165 +0000 UTC m=+840.422012984" watchObservedRunningTime="2026-09-14 13:12:27.131589557 +0000 UTC m=+840.422923368"
Sep 14 13:12:27.132837 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:12:27.132811    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7431-f6537a\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="2b2bc118-676d-49ef-a642-6fad72bbf0cd" pod="opendatahub/test-curl-7431-f6537a"
Sep 14 13:12:27.151637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:27.151520    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54" podStartSLOduration=1.8280801119999999 podStartE2EDuration="3.151506526s" totalImagesPullingTime="1.323426414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:27.150106826 +0000 UTC m=+840.441440648" watchObservedRunningTime="2026-09-14 13:12:27.151506526 +0000 UTC m=+840.442840343"
Sep 14 13:12:27.245517 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:12:27.245484    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7431-f6537a\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="2b2bc118-676d-49ef-a642-6fad72bbf0cd" pod="opendatahub/test-curl-7431-f6537a"
Sep 14 13:12:27.246626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:27.246607    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b2bc118-676d-49ef-a642-6fad72bbf0cd" path="/var/lib/kubelet/pods/2b2bc118-676d-49ef-a642-6fad72bbf0cd/volumes"
Sep 14 13:12:27.511660 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:27.511626    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:28.114610 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:28.114581    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:28.286431 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:28.286402    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7431-327b98"
Sep 14 13:12:28.410775 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:28.410686    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8mjc\" (UniqueName: \"kubernetes.io/projected/518a90b4-f2a8-4a39-944e-50a41f5080dc-kube-api-access-k8mjc\") pod \"518a90b4-f2a8-4a39-944e-50a41f5080dc\" (UID: \"518a90b4-f2a8-4a39-944e-50a41f5080dc\") "
Sep 14 13:12:28.412816 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:28.412790    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/518a90b4-f2a8-4a39-944e-50a41f5080dc-kube-api-access-k8mjc" (OuterVolumeSpecName: "kube-api-access-k8mjc") pod "518a90b4-f2a8-4a39-944e-50a41f5080dc" (UID: "518a90b4-f2a8-4a39-944e-50a41f5080dc"). InnerVolumeSpecName "kube-api-access-k8mjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:12:28.511486 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:28.511441    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8mjc\" (UniqueName: \"kubernetes.io/projected/518a90b4-f2a8-4a39-944e-50a41f5080dc-kube-api-access-k8mjc\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:12:28.735013 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:28.734914    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:28.822919 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:28.822886    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:29.126823 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:29.126781    2581 generic.go:356] "Generic (PLEG): container finished" podID="518a90b4-f2a8-4a39-944e-50a41f5080dc" containerID="0464e75075459bc33ea26fd49c57937d2f842fb9b346c742cc91249464bc3d28" exitCode=137
Sep 14 13:12:29.127292 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:29.126840    2581 scope.go:117] "RemoveContainer" containerID="0464e75075459bc33ea26fd49c57937d2f842fb9b346c742cc91249464bc3d28"
Sep 14 13:12:29.127292 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:29.126841    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7431-327b98"
Sep 14 13:12:29.135830 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:29.135812    2581 scope.go:117] "RemoveContainer" containerID="0464e75075459bc33ea26fd49c57937d2f842fb9b346c742cc91249464bc3d28"
Sep 14 13:12:29.136154 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:12:29.136135    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0464e75075459bc33ea26fd49c57937d2f842fb9b346c742cc91249464bc3d28\": container with ID starting with 0464e75075459bc33ea26fd49c57937d2f842fb9b346c742cc91249464bc3d28 not found: ID does not exist" containerID="0464e75075459bc33ea26fd49c57937d2f842fb9b346c742cc91249464bc3d28"
Sep 14 13:12:29.136250 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:29.136162    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0464e75075459bc33ea26fd49c57937d2f842fb9b346c742cc91249464bc3d28"} err="failed to get container status \"0464e75075459bc33ea26fd49c57937d2f842fb9b346c742cc91249464bc3d28\": rpc error: code = NotFound desc = could not find container \"0464e75075459bc33ea26fd49c57937d2f842fb9b346c742cc91249464bc3d28\": container with ID starting with 0464e75075459bc33ea26fd49c57937d2f842fb9b346c742cc91249464bc3d28 not found: ID does not exist"
Sep 14 13:12:29.137370 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:12:29.137346    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7431-327b98\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="518a90b4-f2a8-4a39-944e-50a41f5080dc" pod="opendatahub/test-curl-7431-327b98"
Sep 14 13:12:29.245959 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:29.245923    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="518a90b4-f2a8-4a39-944e-50a41f5080dc" path="/var/lib/kubelet/pods/518a90b4-f2a8-4a39-944e-50a41f5080dc/volumes"
Sep 14 13:12:29.358741 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:29.358711    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:29.689846 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:29.689811    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:29.785181 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:29.785144    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:30.659826 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:30.659795    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:31.755881 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:31.755851    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:32.656805 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.656771    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"]
Sep 14 13:12:32.657262 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.657243    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="518a90b4-f2a8-4a39-944e-50a41f5080dc" containerName="test-curl-7431-327b98"
Sep 14 13:12:32.657380 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.657265    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="518a90b4-f2a8-4a39-944e-50a41f5080dc" containerName="test-curl-7431-327b98"
Sep 14 13:12:32.657380 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.657345    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="518a90b4-f2a8-4a39-944e-50a41f5080dc" containerName="test-curl-7431-327b98"
Sep 14 13:12:32.661957 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.661931    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:32.664713 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.664665    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-360610b9-serving-cert\""
Sep 14 13:12:32.665115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.665094    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 13:12:32.665217 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.665201    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kgb2c\""
Sep 14 13:12:32.666103 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.665902    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 13:12:32.666103 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.666081    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 13:12:32.674229 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.674200    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"]
Sep 14 13:12:32.703987 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.703947    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:32.754260 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.754224    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rk59\" (UniqueName: \"kubernetes.io/projected/be643d44-d9b5-4c30-9056-d25cbdb52dae-kube-api-access-4rk59\") pod \"maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:32.754423 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.754279    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-maas-api-tls\") pod \"maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:32.754423 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.754304    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-metrics-certs\") pod \"maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:32.754423 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.754322    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be643d44-d9b5-4c30-9056-d25cbdb52dae-openshift-service-ca\") pod \"maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:32.803915 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.803878    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:32.855521 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.855472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rk59\" (UniqueName: \"kubernetes.io/projected/be643d44-d9b5-4c30-9056-d25cbdb52dae-kube-api-access-4rk59\") pod \"maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:32.855729 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.855562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-maas-api-tls\") pod \"maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:32.855729 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.855583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-metrics-certs\") pod \"maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:32.855729 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:12:32.855656    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-serving-cert: secret "maas-api-e2e-ait-360610b9-serving-cert" not found
Sep 14 13:12:32.855729 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.855698    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be643d44-d9b5-4c30-9056-d25cbdb52dae-openshift-service-ca\") pod \"maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:32.855729 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:12:32.855726    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-maas-api-tls podName:be643d44-d9b5-4c30-9056-d25cbdb52dae nodeName:}" failed. No retries permitted until 2026-09-14 13:12:33.355704949 +0000 UTC m=+846.647038758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-maas-api-tls") pod "maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx" (UID: "be643d44-d9b5-4c30-9056-d25cbdb52dae") : secret "maas-api-e2e-ait-360610b9-serving-cert" not found
Sep 14 13:12:32.856331 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.856311    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be643d44-d9b5-4c30-9056-d25cbdb52dae-openshift-service-ca\") pod \"maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:32.858020 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.857999    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-metrics-certs\") pod \"maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:32.865026 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:32.864992    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rk59\" (UniqueName: \"kubernetes.io/projected/be643d44-d9b5-4c30-9056-d25cbdb52dae-kube-api-access-4rk59\") pod \"maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:33.075332 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.075243    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:33.336950 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.336831    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:33.360769 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.360731    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-maas-api-tls\") pod \"maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:33.363205 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.363170    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-maas-api-tls\") pod \"maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:33.487467 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.487429    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:33.575135 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.575099    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:33.621052 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.621013    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"]
Sep 14 13:12:33.627619 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.627592    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"
Sep 14 13:12:33.630645 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.630213    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-360610b9-dockercfg-2rps9\""
Sep 14 13:12:33.630775 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.630754    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-360610b9\""
Sep 14 13:12:33.644008 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.642818    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"]
Sep 14 13:12:33.772828 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.772787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm767\" (UniqueName: \"kubernetes.io/projected/3bfa8355-485f-434e-b1d8-07adbe893c17-kube-api-access-gm767\") pod \"payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng\" (UID: \"3bfa8355-485f-434e-b1d8-07adbe893c17\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"
Sep 14 13:12:33.773050 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.772948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bfa8355-485f-434e-b1d8-07adbe893c17-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng\" (UID: \"3bfa8355-485f-434e-b1d8-07adbe893c17\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"
Sep 14 13:12:33.852283 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.852214    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"]
Sep 14 13:12:33.854456 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:12:33.854427    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe643d44_d9b5_4c30_9056_d25cbdb52dae.slice/crio-be9f10fe229a341965174b4cf6db969f4b3a5429aa72072fea8f2785de59c53d WatchSource:0}: Error finding container be9f10fe229a341965174b4cf6db969f4b3a5429aa72072fea8f2785de59c53d: Status 404 returned error can't find the container with id be9f10fe229a341965174b4cf6db969f4b3a5429aa72072fea8f2785de59c53d
Sep 14 13:12:33.873570 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.873473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bfa8355-485f-434e-b1d8-07adbe893c17-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng\" (UID: \"3bfa8355-485f-434e-b1d8-07adbe893c17\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"
Sep 14 13:12:33.873570 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.873554    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm767\" (UniqueName: \"kubernetes.io/projected/3bfa8355-485f-434e-b1d8-07adbe893c17-kube-api-access-gm767\") pod \"payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng\" (UID: \"3bfa8355-485f-434e-b1d8-07adbe893c17\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"
Sep 14 13:12:33.874233 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.874210    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bfa8355-485f-434e-b1d8-07adbe893c17-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng\" (UID: \"3bfa8355-485f-434e-b1d8-07adbe893c17\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"
Sep 14 13:12:33.883968 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.883945    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm767\" (UniqueName: \"kubernetes.io/projected/3bfa8355-485f-434e-b1d8-07adbe893c17-kube-api-access-gm767\") pod \"payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng\" (UID: \"3bfa8355-485f-434e-b1d8-07adbe893c17\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"
Sep 14 13:12:33.946825 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:33.946788    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"
Sep 14 13:12:34.005985 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:34.005944    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:34.100845 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:34.100809    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:34.153130 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:34.153036    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx" event={"ID":"be643d44-d9b5-4c30-9056-d25cbdb52dae","Type":"ContainerStarted","Data":"be9f10fe229a341965174b4cf6db969f4b3a5429aa72072fea8f2785de59c53d"}
Sep 14 13:12:34.232776 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:34.232746    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"]
Sep 14 13:12:34.235716 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:12:34.235676    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bfa8355_485f_434e_b1d8_07adbe893c17.slice/crio-498e1ce46beed5bda5efa023c1bd9b1089ebfc716065f7bfb38e290e021f66db WatchSource:0}: Error finding container 498e1ce46beed5bda5efa023c1bd9b1089ebfc716065f7bfb38e290e021f66db: Status 404 returned error can't find the container with id 498e1ce46beed5bda5efa023c1bd9b1089ebfc716065f7bfb38e290e021f66db
Sep 14 13:12:34.675067 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:34.675034    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:34.784970 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:34.784924    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:34.983935 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:34.983845    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:35.114947 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.114912    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"]
Sep 14 13:12:35.119296 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.119261    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"
Sep 14 13:12:35.123209 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.123174    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-c14a5e-dockercfg-8mt9k\""
Sep 14 13:12:35.123595 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.123570    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-c14a5e\""
Sep 14 13:12:35.132322 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.132293    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"]
Sep 14 13:12:35.161421 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.161289    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng" event={"ID":"3bfa8355-485f-434e-b1d8-07adbe893c17","Type":"ContainerStarted","Data":"23f37417d3f42ba05cf3ea72e32dd225576b5cd2406c7da6e7ce89e6b454d515"}
Sep 14 13:12:35.161421 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.161356    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"
Sep 14 13:12:35.161421 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.161385    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng" event={"ID":"3bfa8355-485f-434e-b1d8-07adbe893c17","Type":"ContainerStarted","Data":"498e1ce46beed5bda5efa023c1bd9b1089ebfc716065f7bfb38e290e021f66db"}
Sep 14 13:12:35.193132 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.193009    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng" podStartSLOduration=2.192989307 podStartE2EDuration="2.192989307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:35.189683444 +0000 UTC m=+848.481017261" watchObservedRunningTime="2026-09-14 13:12:35.192989307 +0000 UTC m=+848.484323123"
Sep 14 13:12:35.239356 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.239275    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:35.289040 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.288997    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnw2c\" (UniqueName: \"kubernetes.io/projected/05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e-kube-api-access-nnw2c\") pod \"payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns\" (UID: \"05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"
Sep 14 13:12:35.289294 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.289266    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns\" (UID: \"05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"
Sep 14 13:12:35.322135 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.322103    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:35.390013 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.389973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns\" (UID: \"05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"
Sep 14 13:12:35.390219 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.390026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnw2c\" (UniqueName: \"kubernetes.io/projected/05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e-kube-api-access-nnw2c\") pod \"payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns\" (UID: \"05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"
Sep 14 13:12:35.390883 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.390858    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns\" (UID: \"05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"
Sep 14 13:12:35.403890 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.403862    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnw2c\" (UniqueName: \"kubernetes.io/projected/05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e-kube-api-access-nnw2c\") pod \"payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns\" (UID: \"05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"
Sep 14 13:12:35.422615 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.422578    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:35.434467 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.434421    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"
Sep 14 13:12:35.558004 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.557904    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:35.688934 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.688904    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:35.837674 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:35.837592    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:36.023804 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:36.023754    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:36.113687 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:36.113651    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:36.213487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:36.213450    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:36.315189 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:36.315159    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:36.548960 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:36.548926    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:36.564031 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:36.564003    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"]
Sep 14 13:12:36.566225 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:12:36.566196    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05cd1fd5_f01c_47ec_b8a6_f18c014d0a2e.slice/crio-e246ac9551d8cbf8a8fe75932ec564bf44f41660b87f2b22f92fd96321f4b07d WatchSource:0}: Error finding container e246ac9551d8cbf8a8fe75932ec564bf44f41660b87f2b22f92fd96321f4b07d: Status 404 returned error can't find the container with id e246ac9551d8cbf8a8fe75932ec564bf44f41660b87f2b22f92fd96321f4b07d
Sep 14 13:12:37.174173 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:37.174133    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns" event={"ID":"05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e","Type":"ContainerStarted","Data":"397ad8301965d07224b8ff09d47b3253d79d236259b7a81967f3d0b19a92df6a"}
Sep 14 13:12:37.174671 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:37.174280    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"
Sep 14 13:12:37.174671 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:37.174323    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns" event={"ID":"05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e","Type":"ContainerStarted","Data":"e246ac9551d8cbf8a8fe75932ec564bf44f41660b87f2b22f92fd96321f4b07d"}
Sep 14 13:12:37.175740 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:37.175717    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx" event={"ID":"be643d44-d9b5-4c30-9056-d25cbdb52dae","Type":"ContainerStarted","Data":"ff7f4cdc9df68b957769d17e61f65f84e8ca098bb4017766e32dcbc5a853a258"}
Sep 14 13:12:37.175817 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:37.175770    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:37.193611 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:37.193566    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns" podStartSLOduration=2.193551728 podStartE2EDuration="2.193551728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:37.192761549 +0000 UTC m=+850.484095366" watchObservedRunningTime="2026-09-14 13:12:37.193551728 +0000 UTC m=+850.484885539"
Sep 14 13:12:37.217846 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:37.217785    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx" podStartSLOduration=2.7248652030000002 podStartE2EDuration="5.217767533s" totalImagesPullingTime="2.49290233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:37.21629712 +0000 UTC m=+850.507630937" watchObservedRunningTime="2026-09-14 13:12:37.217767533 +0000 UTC m=+850.509101349"
Sep 14 13:12:37.498093 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:37.497997    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:38.122975 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:38.122940    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"
Sep 14 13:12:38.123176 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:38.123158    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"
Sep 14 13:12:38.256061 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:38.256023    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:38.363558 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:38.360452    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:12:38.886697 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:38.886660    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:39.333080 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:39.332974    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:39.418939 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:39.418907    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:40.338058 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:40.338025    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:40.453608 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:40.453576    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:40.735969 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:40.735933    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:40.973478 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:40.973446    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:41.069788 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:41.069705    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:41.560782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:41.560751    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:42.578041 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.578008    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"]
Sep 14 13:12:42.581987 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.581962    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:42.584577 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.584552    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-cadd9c-serving-cert\""
Sep 14 13:12:42.593671 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.593647    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"]
Sep 14 13:12:42.619146 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.619118    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:42.722861 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.722823    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:42.764250 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.764219    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qk4d\" (UniqueName: \"kubernetes.io/projected/ba5f701e-637f-44b4-8b4a-090d42179f7e-kube-api-access-8qk4d\") pod \"maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:42.764421 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.764260    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba5f701e-637f-44b4-8b4a-090d42179f7e-maas-api-tls\") pod \"maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:42.764421 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.764311    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba5f701e-637f-44b4-8b4a-090d42179f7e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:42.764421 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.764349    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba5f701e-637f-44b4-8b4a-090d42179f7e-metrics-certs\") pod \"maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:42.818829 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.818792    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:42.865007 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.864979    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba5f701e-637f-44b4-8b4a-090d42179f7e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:42.865007 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.865009    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba5f701e-637f-44b4-8b4a-090d42179f7e-metrics-certs\") pod \"maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:42.865254 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.865090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qk4d\" (UniqueName: \"kubernetes.io/projected/ba5f701e-637f-44b4-8b4a-090d42179f7e-kube-api-access-8qk4d\") pod \"maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:42.865254 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.865118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba5f701e-637f-44b4-8b4a-090d42179f7e-maas-api-tls\") pod \"maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:42.865755 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.865731    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba5f701e-637f-44b4-8b4a-090d42179f7e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:42.867505 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.867480    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba5f701e-637f-44b4-8b4a-090d42179f7e-metrics-certs\") pod \"maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:42.867623 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.867571    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba5f701e-637f-44b4-8b4a-090d42179f7e-maas-api-tls\") pod \"maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:42.872518 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.872496    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qk4d\" (UniqueName: \"kubernetes.io/projected/ba5f701e-637f-44b4-8b4a-090d42179f7e-kube-api-access-8qk4d\") pod \"maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:42.893428 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:42.893404    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:43.171643 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.171569    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"]
Sep 14 13:12:43.174610 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:12:43.174579    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba5f701e_637f_44b4_8b4a_090d42179f7e.slice/crio-5678fad21acf10a61714c6294e983ece8530290db7204e2ba85e8ddf0d022215 WatchSource:0}: Error finding container 5678fad21acf10a61714c6294e983ece8530290db7204e2ba85e8ddf0d022215: Status 404 returned error can't find the container with id 5678fad21acf10a61714c6294e983ece8530290db7204e2ba85e8ddf0d022215
Sep 14 13:12:43.187822 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.187799    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:12:43.207564 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.207512    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v" event={"ID":"ba5f701e-637f-44b4-8b4a-090d42179f7e","Type":"ContainerStarted","Data":"5678fad21acf10a61714c6294e983ece8530290db7204e2ba85e8ddf0d022215"}
Sep 14 13:12:43.220353 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.220322    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:43.321233 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.321199    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:43.434178 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.434095    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"]
Sep 14 13:12:43.439017 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.438998    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"
Sep 14 13:12:43.441985 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.441961    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-cadd9c\""
Sep 14 13:12:43.442514 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.442496    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-cadd9c-dockercfg-jkcfv\""
Sep 14 13:12:43.447852 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.447831    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"]
Sep 14 13:12:43.470940 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.470902    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af5c1011-5523-466c-b844-787991bce9db-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q\" (UID: \"af5c1011-5523-466c-b844-787991bce9db\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"
Sep 14 13:12:43.471101 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.470944    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns475\" (UniqueName: \"kubernetes.io/projected/af5c1011-5523-466c-b844-787991bce9db-kube-api-access-ns475\") pod \"payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q\" (UID: \"af5c1011-5523-466c-b844-787991bce9db\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"
Sep 14 13:12:43.502510 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.502481    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"]
Sep 14 13:12:43.506295 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.506272    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"
Sep 14 13:12:43.515459 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.515426    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"]
Sep 14 13:12:43.571623 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.571584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af5c1011-5523-466c-b844-787991bce9db-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q\" (UID: \"af5c1011-5523-466c-b844-787991bce9db\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"
Sep 14 13:12:43.571623 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.571630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns475\" (UniqueName: \"kubernetes.io/projected/af5c1011-5523-466c-b844-787991bce9db-kube-api-access-ns475\") pod \"payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q\" (UID: \"af5c1011-5523-466c-b844-787991bce9db\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"
Sep 14 13:12:43.571920 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.571658    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23719951-7853-459f-afed-b3ba7c2b8660-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9\" (UID: \"23719951-7853-459f-afed-b3ba7c2b8660\") " pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"
Sep 14 13:12:43.571920 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.571692    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqr6w\" (UniqueName: \"kubernetes.io/projected/23719951-7853-459f-afed-b3ba7c2b8660-kube-api-access-dqr6w\") pod \"payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9\" (UID: \"23719951-7853-459f-afed-b3ba7c2b8660\") " pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"
Sep 14 13:12:43.572397 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.572371    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af5c1011-5523-466c-b844-787991bce9db-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q\" (UID: \"af5c1011-5523-466c-b844-787991bce9db\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"
Sep 14 13:12:43.579892 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.579864    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns475\" (UniqueName: \"kubernetes.io/projected/af5c1011-5523-466c-b844-787991bce9db-kube-api-access-ns475\") pod \"payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q\" (UID: \"af5c1011-5523-466c-b844-787991bce9db\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"
Sep 14 13:12:43.672637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.672594    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23719951-7853-459f-afed-b3ba7c2b8660-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9\" (UID: \"23719951-7853-459f-afed-b3ba7c2b8660\") " pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"
Sep 14 13:12:43.672637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.672646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqr6w\" (UniqueName: \"kubernetes.io/projected/23719951-7853-459f-afed-b3ba7c2b8660-kube-api-access-dqr6w\") pod \"payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9\" (UID: \"23719951-7853-459f-afed-b3ba7c2b8660\") " pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"
Sep 14 13:12:43.673263 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.673240    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23719951-7853-459f-afed-b3ba7c2b8660-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9\" (UID: \"23719951-7853-459f-afed-b3ba7c2b8660\") " pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"
Sep 14 13:12:43.680862 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.680838    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqr6w\" (UniqueName: \"kubernetes.io/projected/23719951-7853-459f-afed-b3ba7c2b8660-kube-api-access-dqr6w\") pod \"payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9\" (UID: \"23719951-7853-459f-afed-b3ba7c2b8660\") " pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"
Sep 14 13:12:43.751217 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.751190    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"
Sep 14 13:12:43.818953 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:43.818920    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"
Sep 14 13:12:44.027172 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.027144    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"]
Sep 14 13:12:44.029496 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:12:44.029464    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf5c1011_5523_466c_b844_787991bce9db.slice/crio-cef3603e0b1c9e0d734596b3e3bad628cf7425b957d30349677e678bcc3a663e WatchSource:0}: Error finding container cef3603e0b1c9e0d734596b3e3bad628cf7425b957d30349677e678bcc3a663e: Status 404 returned error can't find the container with id cef3603e0b1c9e0d734596b3e3bad628cf7425b957d30349677e678bcc3a663e
Sep 14 13:12:44.099941 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.099917    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"]
Sep 14 13:12:44.101915 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:12:44.101889    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23719951_7853_459f_afed_b3ba7c2b8660.slice/crio-0c32933ae0a2d923dfa94ab5b17b7a1bd34f3d75b9d46b7d3e97bb3ebd18c793 WatchSource:0}: Error finding container 0c32933ae0a2d923dfa94ab5b17b7a1bd34f3d75b9d46b7d3e97bb3ebd18c793: Status 404 returned error can't find the container with id 0c32933ae0a2d923dfa94ab5b17b7a1bd34f3d75b9d46b7d3e97bb3ebd18c793
Sep 14 13:12:44.118697 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.118666    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:44.216212 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.216181    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q" event={"ID":"af5c1011-5523-466c-b844-787991bce9db","Type":"ContainerStarted","Data":"0e484f3566f0fbc8c03b28c1c0ffb9e7694b3f547788724f93c565939a17e846"}
Sep 14 13:12:44.216406 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.216241    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q" event={"ID":"af5c1011-5523-466c-b844-787991bce9db","Type":"ContainerStarted","Data":"cef3603e0b1c9e0d734596b3e3bad628cf7425b957d30349677e678bcc3a663e"}
Sep 14 13:12:44.216406 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.216303    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"
Sep 14 13:12:44.217915 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.217889    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v" event={"ID":"ba5f701e-637f-44b4-8b4a-090d42179f7e","Type":"ContainerStarted","Data":"a3e124e7a3bdf0b35e42172bd246ac56cf6a4ade59bf6b48a3230c88f6a401ec"}
Sep 14 13:12:44.218037 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.217989    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:44.219580 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.219556    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9" event={"ID":"23719951-7853-459f-afed-b3ba7c2b8660","Type":"ContainerStarted","Data":"c1c755b4f86a7b199361b610eefcea49a2650e8cc95a82be257ff0b8423dffd0"}
Sep 14 13:12:44.219695 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.219603    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9" event={"ID":"23719951-7853-459f-afed-b3ba7c2b8660","Type":"ContainerStarted","Data":"0c32933ae0a2d923dfa94ab5b17b7a1bd34f3d75b9d46b7d3e97bb3ebd18c793"}
Sep 14 13:12:44.219695 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.219642    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"
Sep 14 13:12:44.236115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.236052    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q" podStartSLOduration=1.236031361 podStartE2EDuration="1.236031361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:44.234123375 +0000 UTC m=+857.525457193" watchObservedRunningTime="2026-09-14 13:12:44.236031361 +0000 UTC m=+857.527365178"
Sep 14 13:12:44.251441 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.251368    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v" podStartSLOduration=1.750619195 podStartE2EDuration="2.251349374s" totalImagesPullingTime="500.730179ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:44.250731168 +0000 UTC m=+857.542064986" watchObservedRunningTime="2026-09-14 13:12:44.251349374 +0000 UTC m=+857.542683195"
Sep 14 13:12:44.267875 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.267821    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9" podStartSLOduration=1.267801033 podStartE2EDuration="1.267801033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:44.265562488 +0000 UTC m=+857.556896309" watchObservedRunningTime="2026-09-14 13:12:44.267801033 +0000 UTC m=+857.559134856"
Sep 14 13:12:44.796139 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.796093    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:44.899224 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.899180    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:44.995800 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:44.995752    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:45.221940 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:45.221904    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:46.170474 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:46.170440    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"
Sep 14 13:12:46.198392 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:46.198361    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:47.172460 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:47.172415    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:47.848402 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:47.848362    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:48.184345 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:48.184314    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"
Sep 14 13:12:48.641288 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:48.641252    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:49.433830 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:49.433794    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:50.231202 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:50.231168    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:12:50.318312 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:50.318283    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:51.118162 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:51.118119    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:51.623613 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:51.623586    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:51.683101 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:51.683068    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:51.763855 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:51.763819    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:51.861995 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:51.861961    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:51.966865 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:51.966776    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:52.066807 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:52.066775    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:52.855877 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:52.855840    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:53.641131 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:53.641101    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:54.351965 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:54.351931    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:12:54.434702 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:54.434673    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:55.227929 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:55.227899    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"
Sep 14 13:12:55.228149 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:55.227983    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"
Sep 14 13:12:55.450840 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:55.450811    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:56.155867 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:56.155833    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:57.063171 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:57.063138    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:57.764431 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:57.764391    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:58.175579 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:58.175509    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:58.282902 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:58.282869    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:58.492253 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:58.492167    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:58.566240 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:58.566202    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:58.624896 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:58.624861    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:58.732492 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:58.732457    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:58.960380 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:58.960346    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:59.275790 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:59.275710    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:59.929141 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:12:59.929104    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:00.516255 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.516218    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"]
Sep 14 13:13:00.516775 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.516466    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx" podUID="be643d44-d9b5-4c30-9056-d25cbdb52dae" containerName="maas-api" containerID="cri-o://ff7f4cdc9df68b957769d17e61f65f84e8ca098bb4017766e32dcbc5a853a258" gracePeriod=30
Sep 14 13:13:00.544164 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.544132    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:00.578716 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.578686    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"]
Sep 14 13:13:00.578956 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.578903    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng" podUID="3bfa8355-485f-434e-b1d8-07adbe893c17" containerName="payload-pre-processing" containerID="cri-o://23f37417d3f42ba05cf3ea72e32dd225576b5cd2406c7da6e7ce89e6b454d515" gracePeriod=30
Sep 14 13:13:00.644339 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.644309    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:00.724924 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.724880    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:00.783469 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.783398    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:00.843224 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.843195    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:13:00.845961 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.845936    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"
Sep 14 13:13:00.933433 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.933391    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rk59\" (UniqueName: \"kubernetes.io/projected/be643d44-d9b5-4c30-9056-d25cbdb52dae-kube-api-access-4rk59\") pod \"be643d44-d9b5-4c30-9056-d25cbdb52dae\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") "
Sep 14 13:13:00.933433 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.933444    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-metrics-certs\") pod \"be643d44-d9b5-4c30-9056-d25cbdb52dae\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") "
Sep 14 13:13:00.933680 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.933480    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-maas-api-tls\") pod \"be643d44-d9b5-4c30-9056-d25cbdb52dae\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") "
Sep 14 13:13:00.933680 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.933499    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bfa8355-485f-434e-b1d8-07adbe893c17-plugins-config-volume\") pod \"3bfa8355-485f-434e-b1d8-07adbe893c17\" (UID: \"3bfa8355-485f-434e-b1d8-07adbe893c17\") "
Sep 14 13:13:00.933680 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.933523    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be643d44-d9b5-4c30-9056-d25cbdb52dae-openshift-service-ca\") pod \"be643d44-d9b5-4c30-9056-d25cbdb52dae\" (UID: \"be643d44-d9b5-4c30-9056-d25cbdb52dae\") "
Sep 14 13:13:00.933680 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.933548    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm767\" (UniqueName: \"kubernetes.io/projected/3bfa8355-485f-434e-b1d8-07adbe893c17-kube-api-access-gm767\") pod \"3bfa8355-485f-434e-b1d8-07adbe893c17\" (UID: \"3bfa8355-485f-434e-b1d8-07adbe893c17\") "
Sep 14 13:13:00.933970 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.933921    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be643d44-d9b5-4c30-9056-d25cbdb52dae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "be643d44-d9b5-4c30-9056-d25cbdb52dae" (UID: "be643d44-d9b5-4c30-9056-d25cbdb52dae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:13:00.934050 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.933972    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bfa8355-485f-434e-b1d8-07adbe893c17-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3bfa8355-485f-434e-b1d8-07adbe893c17" (UID: "3bfa8355-485f-434e-b1d8-07adbe893c17"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:13:00.935812 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.935784    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "be643d44-d9b5-4c30-9056-d25cbdb52dae" (UID: "be643d44-d9b5-4c30-9056-d25cbdb52dae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:13:00.935922 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.935862    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be643d44-d9b5-4c30-9056-d25cbdb52dae-kube-api-access-4rk59" (OuterVolumeSpecName: "kube-api-access-4rk59") pod "be643d44-d9b5-4c30-9056-d25cbdb52dae" (UID: "be643d44-d9b5-4c30-9056-d25cbdb52dae"). InnerVolumeSpecName "kube-api-access-4rk59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:13:00.936083 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.936056    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "be643d44-d9b5-4c30-9056-d25cbdb52dae" (UID: "be643d44-d9b5-4c30-9056-d25cbdb52dae"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:13:00.936180 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.936085    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bfa8355-485f-434e-b1d8-07adbe893c17-kube-api-access-gm767" (OuterVolumeSpecName: "kube-api-access-gm767") pod "3bfa8355-485f-434e-b1d8-07adbe893c17" (UID: "3bfa8355-485f-434e-b1d8-07adbe893c17"). InnerVolumeSpecName "kube-api-access-gm767". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:13:00.941412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:00.941389    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:01.034210 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.034137    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rk59\" (UniqueName: \"kubernetes.io/projected/be643d44-d9b5-4c30-9056-d25cbdb52dae-kube-api-access-4rk59\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:13:01.034210 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.034160    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-metrics-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:13:01.034210 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.034170    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be643d44-d9b5-4c30-9056-d25cbdb52dae-maas-api-tls\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:13:01.034210 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.034179    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bfa8355-485f-434e-b1d8-07adbe893c17-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:13:01.034210 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.034190    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be643d44-d9b5-4c30-9056-d25cbdb52dae-openshift-service-ca\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:13:01.034462 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.034220    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm767\" (UniqueName: \"kubernetes.io/projected/3bfa8355-485f-434e-b1d8-07adbe893c17-kube-api-access-gm767\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:13:01.141949 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.141909    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:01.307877 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.307786    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:01.317394 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.317365    2581 generic.go:356] "Generic (PLEG): container finished" podID="be643d44-d9b5-4c30-9056-d25cbdb52dae" containerID="ff7f4cdc9df68b957769d17e61f65f84e8ca098bb4017766e32dcbc5a853a258" exitCode=0
Sep 14 13:13:01.317525 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.317415    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"
Sep 14 13:13:01.317525 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.317442    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx" event={"ID":"be643d44-d9b5-4c30-9056-d25cbdb52dae","Type":"ContainerDied","Data":"ff7f4cdc9df68b957769d17e61f65f84e8ca098bb4017766e32dcbc5a853a258"}
Sep 14 13:13:01.317525 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.317495    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx" event={"ID":"be643d44-d9b5-4c30-9056-d25cbdb52dae","Type":"ContainerDied","Data":"be9f10fe229a341965174b4cf6db969f4b3a5429aa72072fea8f2785de59c53d"}
Sep 14 13:13:01.317525 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.317518    2581 scope.go:117] "RemoveContainer" containerID="ff7f4cdc9df68b957769d17e61f65f84e8ca098bb4017766e32dcbc5a853a258"
Sep 14 13:13:01.319104 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.319080    2581 generic.go:356] "Generic (PLEG): container finished" podID="3bfa8355-485f-434e-b1d8-07adbe893c17" containerID="23f37417d3f42ba05cf3ea72e32dd225576b5cd2406c7da6e7ce89e6b454d515" exitCode=0
Sep 14 13:13:01.319227 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.319125    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng" event={"ID":"3bfa8355-485f-434e-b1d8-07adbe893c17","Type":"ContainerDied","Data":"23f37417d3f42ba05cf3ea72e32dd225576b5cd2406c7da6e7ce89e6b454d515"}
Sep 14 13:13:01.319227 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.319140    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"
Sep 14 13:13:01.319227 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.319167    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng" event={"ID":"3bfa8355-485f-434e-b1d8-07adbe893c17","Type":"ContainerDied","Data":"498e1ce46beed5bda5efa023c1bd9b1089ebfc716065f7bfb38e290e021f66db"}
Sep 14 13:13:01.328259 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.328243    2581 scope.go:117] "RemoveContainer" containerID="ff7f4cdc9df68b957769d17e61f65f84e8ca098bb4017766e32dcbc5a853a258"
Sep 14 13:13:01.328571 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:13:01.328549    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff7f4cdc9df68b957769d17e61f65f84e8ca098bb4017766e32dcbc5a853a258\": container with ID starting with ff7f4cdc9df68b957769d17e61f65f84e8ca098bb4017766e32dcbc5a853a258 not found: ID does not exist" containerID="ff7f4cdc9df68b957769d17e61f65f84e8ca098bb4017766e32dcbc5a853a258"
Sep 14 13:13:01.328626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.328579    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff7f4cdc9df68b957769d17e61f65f84e8ca098bb4017766e32dcbc5a853a258"} err="failed to get container status \"ff7f4cdc9df68b957769d17e61f65f84e8ca098bb4017766e32dcbc5a853a258\": rpc error: code = NotFound desc = could not find container \"ff7f4cdc9df68b957769d17e61f65f84e8ca098bb4017766e32dcbc5a853a258\": container with ID starting with ff7f4cdc9df68b957769d17e61f65f84e8ca098bb4017766e32dcbc5a853a258 not found: ID does not exist"
Sep 14 13:13:01.328626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.328595    2581 scope.go:117] "RemoveContainer" containerID="23f37417d3f42ba05cf3ea72e32dd225576b5cd2406c7da6e7ce89e6b454d515"
Sep 14 13:13:01.336029 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.336012    2581 scope.go:117] "RemoveContainer" containerID="23f37417d3f42ba05cf3ea72e32dd225576b5cd2406c7da6e7ce89e6b454d515"
Sep 14 13:13:01.336310 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:13:01.336288    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23f37417d3f42ba05cf3ea72e32dd225576b5cd2406c7da6e7ce89e6b454d515\": container with ID starting with 23f37417d3f42ba05cf3ea72e32dd225576b5cd2406c7da6e7ce89e6b454d515 not found: ID does not exist" containerID="23f37417d3f42ba05cf3ea72e32dd225576b5cd2406c7da6e7ce89e6b454d515"
Sep 14 13:13:01.336387 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.336316    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23f37417d3f42ba05cf3ea72e32dd225576b5cd2406c7da6e7ce89e6b454d515"} err="failed to get container status \"23f37417d3f42ba05cf3ea72e32dd225576b5cd2406c7da6e7ce89e6b454d515\": rpc error: code = NotFound desc = could not find container \"23f37417d3f42ba05cf3ea72e32dd225576b5cd2406c7da6e7ce89e6b454d515\": container with ID starting with 23f37417d3f42ba05cf3ea72e32dd225576b5cd2406c7da6e7ce89e6b454d515 not found: ID does not exist"
Sep 14 13:13:01.341989 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.341970    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"]
Sep 14 13:13:01.348323 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.348293    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-360610b9-d7d9f7c8b-h9fng"]
Sep 14 13:13:01.358624 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.358604    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"]
Sep 14 13:13:01.360688 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.360666    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-360610b9-7d5fd55878-hjwlx"]
Sep 14 13:13:01.451341 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.451311    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:01.551969 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.551941    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:01.820052 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:01.820010    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:02.070965 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:02.070867    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:02.196733 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:02.196700    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:02.836195 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:02.836159    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:03.245968 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:03.245937    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bfa8355-485f-434e-b1d8-07adbe893c17" path="/var/lib/kubelet/pods/3bfa8355-485f-434e-b1d8-07adbe893c17/volumes"
Sep 14 13:13:03.246346 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:03.246332    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be643d44-d9b5-4c30-9056-d25cbdb52dae" path="/var/lib/kubelet/pods/be643d44-d9b5-4c30-9056-d25cbdb52dae/volumes"
Sep 14 13:13:03.532851 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:03.532767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:04.242055 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:04.242020    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:04.532310 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:04.532234    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:04.632094 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:04.632065    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:05.057489 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.057449    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:05.729308 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.729273    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:05.791372 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.791338    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:05.887022 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.886989    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"]
Sep 14 13:13:05.887399 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.887386    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bfa8355-485f-434e-b1d8-07adbe893c17" containerName="payload-pre-processing"
Sep 14 13:13:05.887495 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.887402    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bfa8355-485f-434e-b1d8-07adbe893c17" containerName="payload-pre-processing"
Sep 14 13:13:05.887495 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.887422    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be643d44-d9b5-4c30-9056-d25cbdb52dae" containerName="maas-api"
Sep 14 13:13:05.887495 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.887430    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="be643d44-d9b5-4c30-9056-d25cbdb52dae" containerName="maas-api"
Sep 14 13:13:05.887617 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.887524    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bfa8355-485f-434e-b1d8-07adbe893c17" containerName="payload-pre-processing"
Sep 14 13:13:05.887617 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.887557    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="be643d44-d9b5-4c30-9056-d25cbdb52dae" containerName="maas-api"
Sep 14 13:13:05.891971 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.891946    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:05.892958 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.892943    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:05.896631 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.896613    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-bc851d\"/\"simulator-w0-bc851d-kserve-self-signed-certs\""
Sep 14 13:13:05.896727 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.896636    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-bc851d\"/\"openshift-service-ca.crt\""
Sep 14 13:13:05.896727 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.896614    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-bc851d\"/\"kube-root-ca.crt\""
Sep 14 13:13:05.896727 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.896614    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-bc851d\"/\"default-dockercfg-xtjzp\""
Sep 14 13:13:05.902593 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.902569    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"]
Sep 14 13:13:05.979602 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.979507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwgmn\" (UniqueName: \"kubernetes.io/projected/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-kube-api-access-pwgmn\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:05.979602 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.979589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-dshm\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:05.979799 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.979612    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-tmp-dir\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:05.979799 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.979633    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-tls-certs\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:05.979799 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.979733    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-model-cache\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:05.979799 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:05.979767    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-home\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:06.080669 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.080628    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-model-cache\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:06.080861 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.080678    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-home\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:06.080861 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.080707    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwgmn\" (UniqueName: \"kubernetes.io/projected/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-kube-api-access-pwgmn\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:06.080861 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.080744    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-dshm\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:06.080861 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.080761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-tmp-dir\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:06.080861 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.080785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-tls-certs\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:06.081110 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.081075    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-home\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:06.081164 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.081077    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-model-cache\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:06.081164 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.081138    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-tmp-dir\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:06.083057 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.083038    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-dshm\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:06.083442 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.083422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-tls-certs\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:06.089183 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.089158    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwgmn\" (UniqueName: \"kubernetes.io/projected/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-kube-api-access-pwgmn\") pod \"simulator-w0-bc851d-kserve-598f555d86-47dfw\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") " pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:06.202201 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.202165    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:06.204058 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.204030    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:06.303189 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.303154    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:06.373207 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.373172    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"]
Sep 14 13:13:06.377170 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.377153    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"
Sep 14 13:13:06.380558 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.380510    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-ea32f836-dockercfg-8fl8q\""
Sep 14 13:13:06.380682 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.380545    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-ea32f836\""
Sep 14 13:13:06.387583 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.387561    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"]
Sep 14 13:13:06.392423 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.392399    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:06.407022 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.406999    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"]
Sep 14 13:13:06.412898 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.412881    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"
Sep 14 13:13:06.421743 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.421717    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"]
Sep 14 13:13:06.481377 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.481353    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"]
Sep 14 13:13:06.483474 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.483451    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n9m6\" (UniqueName: \"kubernetes.io/projected/46ec8965-794b-48b4-9fc8-a078088e3d42-kube-api-access-7n9m6\") pod \"payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx\" (UID: \"46ec8965-794b-48b4-9fc8-a078088e3d42\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"
Sep 14 13:13:06.483654 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.483517    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf5dn\" (UniqueName: \"kubernetes.io/projected/a89e18ac-409f-42ff-82ba-23b396bfb7cd-kube-api-access-sf5dn\") pod \"payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp\" (UID: \"a89e18ac-409f-42ff-82ba-23b396bfb7cd\") " pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"
Sep 14 13:13:06.483654 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:13:06.483583    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef6e3627_df7d_404f_9e0c_a069dbc5d8f7.slice/crio-db6f9fd4f41262a35373f82606ad4b0acd756b888194016a78c7e700804ed6ad WatchSource:0}: Error finding container db6f9fd4f41262a35373f82606ad4b0acd756b888194016a78c7e700804ed6ad: Status 404 returned error can't find the container with id db6f9fd4f41262a35373f82606ad4b0acd756b888194016a78c7e700804ed6ad
Sep 14 13:13:06.483654 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.483607    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89e18ac-409f-42ff-82ba-23b396bfb7cd-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp\" (UID: \"a89e18ac-409f-42ff-82ba-23b396bfb7cd\") " pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"
Sep 14 13:13:06.483789 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.483735    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46ec8965-794b-48b4-9fc8-a078088e3d42-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx\" (UID: \"46ec8965-794b-48b4-9fc8-a078088e3d42\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"
Sep 14 13:13:06.584691 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.584649    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89e18ac-409f-42ff-82ba-23b396bfb7cd-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp\" (UID: \"a89e18ac-409f-42ff-82ba-23b396bfb7cd\") " pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"
Sep 14 13:13:06.584835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.584785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46ec8965-794b-48b4-9fc8-a078088e3d42-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx\" (UID: \"46ec8965-794b-48b4-9fc8-a078088e3d42\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"
Sep 14 13:13:06.584877 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.584840    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n9m6\" (UniqueName: \"kubernetes.io/projected/46ec8965-794b-48b4-9fc8-a078088e3d42-kube-api-access-7n9m6\") pod \"payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx\" (UID: \"46ec8965-794b-48b4-9fc8-a078088e3d42\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"
Sep 14 13:13:06.584960 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.584920    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf5dn\" (UniqueName: \"kubernetes.io/projected/a89e18ac-409f-42ff-82ba-23b396bfb7cd-kube-api-access-sf5dn\") pod \"payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp\" (UID: \"a89e18ac-409f-42ff-82ba-23b396bfb7cd\") " pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"
Sep 14 13:13:06.585490 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.585468    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89e18ac-409f-42ff-82ba-23b396bfb7cd-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp\" (UID: \"a89e18ac-409f-42ff-82ba-23b396bfb7cd\") " pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"
Sep 14 13:13:06.585623 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.585572    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46ec8965-794b-48b4-9fc8-a078088e3d42-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx\" (UID: \"46ec8965-794b-48b4-9fc8-a078088e3d42\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"
Sep 14 13:13:06.596867 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.596840    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n9m6\" (UniqueName: \"kubernetes.io/projected/46ec8965-794b-48b4-9fc8-a078088e3d42-kube-api-access-7n9m6\") pod \"payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx\" (UID: \"46ec8965-794b-48b4-9fc8-a078088e3d42\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"
Sep 14 13:13:06.597166 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.597150    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf5dn\" (UniqueName: \"kubernetes.io/projected/a89e18ac-409f-42ff-82ba-23b396bfb7cd-kube-api-access-sf5dn\") pod \"payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp\" (UID: \"a89e18ac-409f-42ff-82ba-23b396bfb7cd\") " pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"
Sep 14 13:13:06.690721 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.690690    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"
Sep 14 13:13:06.725149 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.725122    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"
Sep 14 13:13:06.754398 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.753847    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:06.972569 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:06.972492    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"]
Sep 14 13:13:06.974718 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:13:06.974683    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46ec8965_794b_48b4_9fc8_a078088e3d42.slice/crio-8ea4b8406e0f566ec64a46062572216e5c0772c4bc33bef901094b76ec05c0dd WatchSource:0}: Error finding container 8ea4b8406e0f566ec64a46062572216e5c0772c4bc33bef901094b76ec05c0dd: Status 404 returned error can't find the container with id 8ea4b8406e0f566ec64a46062572216e5c0772c4bc33bef901094b76ec05c0dd
Sep 14 13:13:07.026429 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:07.026408    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"]
Sep 14 13:13:07.028703 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:13:07.028681    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda89e18ac_409f_42ff_82ba_23b396bfb7cd.slice/crio-dec0742c809df8675904278044af69bb30777c0d72bfa2e7fd9efc5fd4d46c64 WatchSource:0}: Error finding container dec0742c809df8675904278044af69bb30777c0d72bfa2e7fd9efc5fd4d46c64: Status 404 returned error can't find the container with id dec0742c809df8675904278044af69bb30777c0d72bfa2e7fd9efc5fd4d46c64
Sep 14 13:13:07.355182 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:07.355140    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx" event={"ID":"46ec8965-794b-48b4-9fc8-a078088e3d42","Type":"ContainerStarted","Data":"672ea0588625c7572ab974253af5867fa3a7d790bfa5fbc26aff5d231ec7ebe5"}
Sep 14 13:13:07.355365 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:07.355212    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"
Sep 14 13:13:07.355365 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:07.355239    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx" event={"ID":"46ec8965-794b-48b4-9fc8-a078088e3d42","Type":"ContainerStarted","Data":"8ea4b8406e0f566ec64a46062572216e5c0772c4bc33bef901094b76ec05c0dd"}
Sep 14 13:13:07.356920 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:07.356887    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw" event={"ID":"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7","Type":"ContainerStarted","Data":"8949a8cf0d3ce63cfb62d6f04f7a1573d10889bec869e2816cdbd43b4da23219"}
Sep 14 13:13:07.357036 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:07.356931    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw" event={"ID":"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7","Type":"ContainerStarted","Data":"db6f9fd4f41262a35373f82606ad4b0acd756b888194016a78c7e700804ed6ad"}
Sep 14 13:13:07.358583 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:07.358559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp" event={"ID":"a89e18ac-409f-42ff-82ba-23b396bfb7cd","Type":"ContainerStarted","Data":"dfb1dcbc658f86e6b2894717438286b95eeec41f25a8d5ddc33dacd6859579b4"}
Sep 14 13:13:07.358583 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:07.358598    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp" event={"ID":"a89e18ac-409f-42ff-82ba-23b396bfb7cd","Type":"ContainerStarted","Data":"dec0742c809df8675904278044af69bb30777c0d72bfa2e7fd9efc5fd4d46c64"}
Sep 14 13:13:07.358772 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:07.358644    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"
Sep 14 13:13:07.370402 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:07.370356    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx" podStartSLOduration=1.370339983 podStartE2EDuration="1.370339983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:13:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:13:07.370292678 +0000 UTC m=+880.661626497" watchObservedRunningTime="2026-09-14 13:13:07.370339983 +0000 UTC m=+880.661673801"
Sep 14 13:13:07.386296 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:07.386255    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw" podStartSLOduration=2.386241466 podStartE2EDuration="2.386241466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:13:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:13:07.385199342 +0000 UTC m=+880.676533161" watchObservedRunningTime="2026-09-14 13:13:07.386241466 +0000 UTC m=+880.677575274"
Sep 14 13:13:07.441276 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:07.440951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:07.459277 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:07.459225    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp" podStartSLOduration=1.459209567 podStartE2EDuration="1.459209567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:13:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:13:07.410258919 +0000 UTC m=+880.701592735" watchObservedRunningTime="2026-09-14 13:13:07.459209567 +0000 UTC m=+880.750543384"
Sep 14 13:13:07.460786 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:07.460766    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:13:08.068288 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:08.068252    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:08.754174 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:08.754141    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:09.374506 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:09.374474    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:16.204479 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:16.204436    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:16.204920 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:16.204507    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:16.217486 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:16.217462    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:16.419086 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:16.419055    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:13:18.365632 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:18.365601    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"
Sep 14 13:13:18.366126 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:18.365683    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"
Sep 14 13:13:21.846850 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:21.846763    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:13:25.992308 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:25.992264    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"]
Sep 14 13:13:25.996125 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:25.996100    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:25.998584 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:25.998556    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b3d6c5\"/\"openshift-service-ca.crt\""
Sep 14 13:13:25.999629 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:25.999610    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b3d6c5\"/\"kube-root-ca.crt\""
Sep 14 13:13:25.999762 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:25.999694    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b3d6c5\"/\"default-dockercfg-lpgkm\""
Sep 14 13:13:25.999762 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:25.999743    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b3d6c5\"/\"simulator-w4-b3d6c5-kserve-self-signed-certs\""
Sep 14 13:13:26.004904 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.004877    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"]
Sep 14 13:13:26.169607 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.169562    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hnzv\" (UniqueName: \"kubernetes.io/projected/8b71224c-5355-4319-a590-56105f625d2f-kube-api-access-6hnzv\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.169607 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.169609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-home\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.169847 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.169765    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-dshm\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.169847 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.169815    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-model-cache\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.169962 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.169885    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-tmp-dir\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.169962 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.169912    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b71224c-5355-4319-a590-56105f625d2f-tls-certs\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.271089 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.271002    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-dshm\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.271089 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.271052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-model-cache\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.271315 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.271091    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-tmp-dir\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.271315 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.271114    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b71224c-5355-4319-a590-56105f625d2f-tls-certs\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.271315 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.271138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hnzv\" (UniqueName: \"kubernetes.io/projected/8b71224c-5355-4319-a590-56105f625d2f-kube-api-access-6hnzv\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.271315 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.271163    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-home\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.271545 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.271506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-tmp-dir\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.271613 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.271558    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-model-cache\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.271845 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.271822    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-home\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.273191 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.273172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-dshm\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.273938 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.273920    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b71224c-5355-4319-a590-56105f625d2f-tls-certs\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.279595 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.279569    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hnzv\" (UniqueName: \"kubernetes.io/projected/8b71224c-5355-4319-a590-56105f625d2f-kube-api-access-6hnzv\") pod \"simulator-w4-b3d6c5-kserve-fc46456db-kh4t4\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") " pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.305839 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.305809    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:26.307713 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.307694    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:26.593690 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:26.593664    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"]
Sep 14 13:13:26.596461 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:13:26.596421    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b71224c_5355_4319_a590_56105f625d2f.slice/crio-35b723b1a81500895b638edda80c45fc98fcabc60b86b2501562f54d456a1e9d WatchSource:0}: Error finding container 35b723b1a81500895b638edda80c45fc98fcabc60b86b2501562f54d456a1e9d: Status 404 returned error can't find the container with id 35b723b1a81500895b638edda80c45fc98fcabc60b86b2501562f54d456a1e9d
Sep 14 13:13:27.183721 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:27.183690    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:13:27.184682 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:27.184664    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:13:27.463609 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:27.463499    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4" event={"ID":"8b71224c-5355-4319-a590-56105f625d2f","Type":"ContainerStarted","Data":"8303058cab88631662c57bb0967f709e01d16fc10b3fe7b981ae88974956038c"}
Sep 14 13:13:27.463609 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:27.463559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4" event={"ID":"8b71224c-5355-4319-a590-56105f625d2f","Type":"ContainerStarted","Data":"35b723b1a81500895b638edda80c45fc98fcabc60b86b2501562f54d456a1e9d"}
Sep 14 13:13:27.491061 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:27.491000    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4" podStartSLOduration=2.490978366 podStartE2EDuration="2.490978366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:13:27.487383406 +0000 UTC m=+900.778717223" watchObservedRunningTime="2026-09-14 13:13:27.490978366 +0000 UTC m=+900.782312183"
Sep 14 13:13:28.156862 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:28.156828    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:29.222816 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:29.222784    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:29.986893 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:29.986857    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:30.746374 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:30.746340    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:31.557790 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:31.557754    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:32.290424 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:32.290388    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:33.068430 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:33.068400    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:33.833266 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:33.833232    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:34.445579 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:34.445541    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:34.769039 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:34.768934    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:13:35.653878 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:35.653846    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:36.308940 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.308901    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:36.309135 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.308983    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:36.320034 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.320008    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:36.322103 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.322070    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:36.357965 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.357918    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"]
Sep 14 13:13:36.358500 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.358436    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp" podUID="a89e18ac-409f-42ff-82ba-23b396bfb7cd" containerName="payload-processing" containerID="cri-o://dfb1dcbc658f86e6b2894717438286b95eeec41f25a8d5ddc33dacd6859579b4" gracePeriod=30
Sep 14 13:13:36.362682 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.362656    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"]
Sep 14 13:13:36.363280 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.363249    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx" podUID="46ec8965-794b-48b4-9fc8-a078088e3d42" containerName="payload-pre-processing" containerID="cri-o://672ea0588625c7572ab974253af5867fa3a7d790bfa5fbc26aff5d231ec7ebe5" gracePeriod=30
Sep 14 13:13:36.402819 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.402764    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:36.488585 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.488523    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:36.520606 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.520573    2581 generic.go:356] "Generic (PLEG): container finished" podID="a89e18ac-409f-42ff-82ba-23b396bfb7cd" containerID="dfb1dcbc658f86e6b2894717438286b95eeec41f25a8d5ddc33dacd6859579b4" exitCode=0
Sep 14 13:13:36.520772 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.520661    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp" event={"ID":"a89e18ac-409f-42ff-82ba-23b396bfb7cd","Type":"ContainerDied","Data":"dfb1dcbc658f86e6b2894717438286b95eeec41f25a8d5ddc33dacd6859579b4"}
Sep 14 13:13:36.522380 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.522347    2581 generic.go:356] "Generic (PLEG): container finished" podID="46ec8965-794b-48b4-9fc8-a078088e3d42" containerID="672ea0588625c7572ab974253af5867fa3a7d790bfa5fbc26aff5d231ec7ebe5" exitCode=0
Sep 14 13:13:36.522550 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.522426    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx" event={"ID":"46ec8965-794b-48b4-9fc8-a078088e3d42","Type":"ContainerDied","Data":"672ea0588625c7572ab974253af5867fa3a7d790bfa5fbc26aff5d231ec7ebe5"}
Sep 14 13:13:36.538319 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.538003    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:13:36.547187 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.547153    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:36.650369 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.650340    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"
Sep 14 13:13:36.667327 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.667256    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:36.668673 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.668647    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sf5dn\" (UniqueName: \"kubernetes.io/projected/a89e18ac-409f-42ff-82ba-23b396bfb7cd-kube-api-access-sf5dn\") pod \"a89e18ac-409f-42ff-82ba-23b396bfb7cd\" (UID: \"a89e18ac-409f-42ff-82ba-23b396bfb7cd\") "
Sep 14 13:13:36.668779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.668761    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89e18ac-409f-42ff-82ba-23b396bfb7cd-plugins-config-volume\") pod \"a89e18ac-409f-42ff-82ba-23b396bfb7cd\" (UID: \"a89e18ac-409f-42ff-82ba-23b396bfb7cd\") "
Sep 14 13:13:36.669679 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.669396    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a89e18ac-409f-42ff-82ba-23b396bfb7cd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a89e18ac-409f-42ff-82ba-23b396bfb7cd" (UID: "a89e18ac-409f-42ff-82ba-23b396bfb7cd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:13:36.669679 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.669650    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89e18ac-409f-42ff-82ba-23b396bfb7cd-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:13:36.671578 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.671544    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a89e18ac-409f-42ff-82ba-23b396bfb7cd-kube-api-access-sf5dn" (OuterVolumeSpecName: "kube-api-access-sf5dn") pod "a89e18ac-409f-42ff-82ba-23b396bfb7cd" (UID: "a89e18ac-409f-42ff-82ba-23b396bfb7cd"). InnerVolumeSpecName "kube-api-access-sf5dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:13:36.672720 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.672653    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"
Sep 14 13:13:36.770373 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.770343    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46ec8965-794b-48b4-9fc8-a078088e3d42-plugins-config-volume\") pod \"46ec8965-794b-48b4-9fc8-a078088e3d42\" (UID: \"46ec8965-794b-48b4-9fc8-a078088e3d42\") "
Sep 14 13:13:36.770598 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.770391    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7n9m6\" (UniqueName: \"kubernetes.io/projected/46ec8965-794b-48b4-9fc8-a078088e3d42-kube-api-access-7n9m6\") pod \"46ec8965-794b-48b4-9fc8-a078088e3d42\" (UID: \"46ec8965-794b-48b4-9fc8-a078088e3d42\") "
Sep 14 13:13:36.770690 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.770643    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sf5dn\" (UniqueName: \"kubernetes.io/projected/a89e18ac-409f-42ff-82ba-23b396bfb7cd-kube-api-access-sf5dn\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:13:36.770806 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.770780    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46ec8965-794b-48b4-9fc8-a078088e3d42-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "46ec8965-794b-48b4-9fc8-a078088e3d42" (UID: "46ec8965-794b-48b4-9fc8-a078088e3d42"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:13:36.772443 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.772426    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46ec8965-794b-48b4-9fc8-a078088e3d42-kube-api-access-7n9m6" (OuterVolumeSpecName: "kube-api-access-7n9m6") pod "46ec8965-794b-48b4-9fc8-a078088e3d42" (UID: "46ec8965-794b-48b4-9fc8-a078088e3d42"). InnerVolumeSpecName "kube-api-access-7n9m6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:13:36.871238 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.871203    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46ec8965-794b-48b4-9fc8-a078088e3d42-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:13:36.871238 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.871233    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7n9m6\" (UniqueName: \"kubernetes.io/projected/46ec8965-794b-48b4-9fc8-a078088e3d42-kube-api-access-7n9m6\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:13:36.946742 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:36.946707    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:37.104178 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:37.104145    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:37.409398 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:37.409357    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:37.528636 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:37.528546    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"
Sep 14 13:13:37.528636 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:37.528586    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx" event={"ID":"46ec8965-794b-48b4-9fc8-a078088e3d42","Type":"ContainerDied","Data":"8ea4b8406e0f566ec64a46062572216e5c0772c4bc33bef901094b76ec05c0dd"}
Sep 14 13:13:37.528858 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:37.528670    2581 scope.go:117] "RemoveContainer" containerID="672ea0588625c7572ab974253af5867fa3a7d790bfa5fbc26aff5d231ec7ebe5"
Sep 14 13:13:37.530175 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:37.530147    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"
Sep 14 13:13:37.530240 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:37.530178    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp" event={"ID":"a89e18ac-409f-42ff-82ba-23b396bfb7cd","Type":"ContainerDied","Data":"dec0742c809df8675904278044af69bb30777c0d72bfa2e7fd9efc5fd4d46c64"}
Sep 14 13:13:37.538898 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:37.538854    2581 scope.go:117] "RemoveContainer" containerID="dfb1dcbc658f86e6b2894717438286b95eeec41f25a8d5ddc33dacd6859579b4"
Sep 14 13:13:37.551193 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:37.551159    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"]
Sep 14 13:13:37.558837 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:37.558804    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-ea32f836-5564f99954-9b4rx"]
Sep 14 13:13:37.569575 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:37.569507    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"]
Sep 14 13:13:37.576870 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:37.576835    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ea32f836-c5d6f4454-sjgzp"]
Sep 14 13:13:37.609813 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:37.609774    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:37.710709 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:37.710667    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:38.541637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:38.541606    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:39.246631 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:39.246599    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46ec8965-794b-48b4-9fc8-a078088e3d42" path="/var/lib/kubelet/pods/46ec8965-794b-48b4-9fc8-a078088e3d42/volumes"
Sep 14 13:13:39.247020 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:39.246963    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a89e18ac-409f-42ff-82ba-23b396bfb7cd" path="/var/lib/kubelet/pods/a89e18ac-409f-42ff-82ba-23b396bfb7cd/volumes"
Sep 14 13:13:39.345296 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:39.345260    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:39.951049 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:39.951016    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:40.054462 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:40.054431    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:40.155630 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:40.155600    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:40.844647 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:40.844616    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:41.504327 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.504286    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"]
Sep 14 13:13:41.504931 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.504910    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46ec8965-794b-48b4-9fc8-a078088e3d42" containerName="payload-pre-processing"
Sep 14 13:13:41.504931 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.504932    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="46ec8965-794b-48b4-9fc8-a078088e3d42" containerName="payload-pre-processing"
Sep 14 13:13:41.505112 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.504949    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a89e18ac-409f-42ff-82ba-23b396bfb7cd" containerName="payload-processing"
Sep 14 13:13:41.505112 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.504958    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a89e18ac-409f-42ff-82ba-23b396bfb7cd" containerName="payload-processing"
Sep 14 13:13:41.505112 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.505051    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a89e18ac-409f-42ff-82ba-23b396bfb7cd" containerName="payload-processing"
Sep 14 13:13:41.505112 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.505062    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="46ec8965-794b-48b4-9fc8-a078088e3d42" containerName="payload-pre-processing"
Sep 14 13:13:41.509946 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.509922    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:41.512284 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.512262    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-dd15e28a-serving-cert\""
Sep 14 13:13:41.521184 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.521155    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"]
Sep 14 13:13:41.535455 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.535429    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:41.618765 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.618731    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e1417a7-5c51-45af-8213-36db030d27a0-openshift-service-ca\") pod \"maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:41.618949 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.618800    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-metrics-certs\") pod \"maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:41.618949 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.618824    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qdcx\" (UniqueName: \"kubernetes.io/projected/4e1417a7-5c51-45af-8213-36db030d27a0-kube-api-access-9qdcx\") pod \"maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:41.618949 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.618868    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-maas-api-tls\") pod \"maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:41.636612 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.636573    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:41.720378 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.720346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e1417a7-5c51-45af-8213-36db030d27a0-openshift-service-ca\") pod \"maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:41.720589 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.720394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-metrics-certs\") pod \"maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:41.720589 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.720412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qdcx\" (UniqueName: \"kubernetes.io/projected/4e1417a7-5c51-45af-8213-36db030d27a0-kube-api-access-9qdcx\") pod \"maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:41.720589 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.720570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-maas-api-tls\") pod \"maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:41.720763 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:13:41.720698    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-serving-cert: secret "maas-api-e2e-ait-dd15e28a-serving-cert" not found
Sep 14 13:13:41.720815 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:13:41.720765    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-maas-api-tls podName:4e1417a7-5c51-45af-8213-36db030d27a0 nodeName:}" failed. No retries permitted until 2026-09-14 13:13:42.220745339 +0000 UTC m=+915.512079145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-maas-api-tls") pod "maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf" (UID: "4e1417a7-5c51-45af-8213-36db030d27a0") : secret "maas-api-e2e-ait-dd15e28a-serving-cert" not found
Sep 14 13:13:41.721112 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.721093    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e1417a7-5c51-45af-8213-36db030d27a0-openshift-service-ca\") pod \"maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:41.722985 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.722966    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-metrics-certs\") pod \"maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:41.728387 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.728366    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qdcx\" (UniqueName: \"kubernetes.io/projected/4e1417a7-5c51-45af-8213-36db030d27a0-kube-api-access-9qdcx\") pod \"maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:41.845282 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.845188    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:41.976803 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:41.976772    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:42.074873 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:42.074836    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:42.225646 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:42.225604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-maas-api-tls\") pod \"maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:42.228147 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:42.228113    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-maas-api-tls\") pod \"maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:42.422750 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:42.422712    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:42.440322 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:42.440292    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:42.726051 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:42.726021    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"]
Sep 14 13:13:42.727556 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:13:42.727521    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e1417a7_5c51_45af_8213_36db030d27a0.slice/crio-3e4d5d4fa47ed9ffc2df126156be35fcd3fe451d8b6c7cfd6b647c0bd37bffc3 WatchSource:0}: Error finding container 3e4d5d4fa47ed9ffc2df126156be35fcd3fe451d8b6c7cfd6b647c0bd37bffc3: Status 404 returned error can't find the container with id 3e4d5d4fa47ed9ffc2df126156be35fcd3fe451d8b6c7cfd6b647c0bd37bffc3
Sep 14 13:13:43.233667 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:43.233632    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:43.563150 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:43.563053    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf" event={"ID":"4e1417a7-5c51-45af-8213-36db030d27a0","Type":"ContainerStarted","Data":"928904b12c864235e09accb0bdfc69762056b36586c09c80d8343a9028b280ee"}
Sep 14 13:13:43.563150 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:43.563130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf" event={"ID":"4e1417a7-5c51-45af-8213-36db030d27a0","Type":"ContainerStarted","Data":"3e4d5d4fa47ed9ffc2df126156be35fcd3fe451d8b6c7cfd6b647c0bd37bffc3"}
Sep 14 13:13:43.563378 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:43.563169    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:43.583333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:43.583276    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf" podStartSLOduration=2.196997707 podStartE2EDuration="2.583261127s" totalImagesPullingTime="386.26342ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:13:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:13:43.581263214 +0000 UTC m=+916.872597042" watchObservedRunningTime="2026-09-14 13:13:43.583261127 +0000 UTC m=+916.874594945"
Sep 14 13:13:43.873993 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:43.873956    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:44.748121 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:44.748086    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:48.227830 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:48.227796    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:49.573968 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:49.573939    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:13:50.753015 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:50.752969    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:13:52.247382 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:52.247343    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:53.496058 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.496027    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"]
Sep 14 13:13:53.499666 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.499637    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.503450 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.503427    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-975227\"/\"openshift-service-ca.crt\""
Sep 14 13:13:53.503631 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.503606    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-975227\"/\"test-model-975227-kserve-self-signed-certs\""
Sep 14 13:13:53.503859 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.503650    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-975227\"/\"kube-root-ca.crt\""
Sep 14 13:13:53.504024 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.503697    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-975227\"/\"default-dockercfg-hqdf4\""
Sep 14 13:13:53.510130 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.510108    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"]
Sep 14 13:13:53.640280 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.640237    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-dshm\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.640476 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.640305    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-model-cache\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.640476 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.640329    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-tmp-dir\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.640476 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.640375    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-home\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.640645 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.640488    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdj2h\" (UniqueName: \"kubernetes.io/projected/febdafa3-b3e3-41ec-820f-b1fcde95d32c-kube-api-access-mdj2h\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.640645 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.640521    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/febdafa3-b3e3-41ec-820f-b1fcde95d32c-tls-certs\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.741312 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.741266    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-model-cache\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.741312 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.741314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-tmp-dir\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.741593 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.741382    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-home\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.741593 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.741424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdj2h\" (UniqueName: \"kubernetes.io/projected/febdafa3-b3e3-41ec-820f-b1fcde95d32c-kube-api-access-mdj2h\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.741593 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.741454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/febdafa3-b3e3-41ec-820f-b1fcde95d32c-tls-certs\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.741593 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.741495    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-dshm\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.741767 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.741743    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-model-cache\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.741823 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.741800    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-tmp-dir\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.741870 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.741824    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-home\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.743817 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.743801    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-dshm\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.743907 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.743889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/febdafa3-b3e3-41ec-820f-b1fcde95d32c-tls-certs\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.749798 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.749734    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdj2h\" (UniqueName: \"kubernetes.io/projected/febdafa3-b3e3-41ec-820f-b1fcde95d32c-kube-api-access-mdj2h\") pod \"test-model-975227-kserve-57cf5c6585-h5v76\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") " pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.810614 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.810575    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:13:53.898365 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:53.898320    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:54.086198 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:54.086107    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"]
Sep 14 13:13:54.089363 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:13:54.089333    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfebdafa3_b3e3_41ec_820f_b1fcde95d32c.slice/crio-444c6e6e25f2a5c0b47689963a61494d212c31279acf59118b1faae2fd610152 WatchSource:0}: Error finding container 444c6e6e25f2a5c0b47689963a61494d212c31279acf59118b1faae2fd610152: Status 404 returned error can't find the container with id 444c6e6e25f2a5c0b47689963a61494d212c31279acf59118b1faae2fd610152
Sep 14 13:13:54.623063 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:54.623022    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76" event={"ID":"febdafa3-b3e3-41ec-820f-b1fcde95d32c","Type":"ContainerStarted","Data":"7a96b8702551fd25e3ca11f177caa0a5c9381efe78e5082e382825d661674a3c"}
Sep 14 13:13:54.623468 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:54.623074    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76" event={"ID":"febdafa3-b3e3-41ec-820f-b1fcde95d32c","Type":"ContainerStarted","Data":"444c6e6e25f2a5c0b47689963a61494d212c31279acf59118b1faae2fd610152"}
Sep 14 13:13:54.640770 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:13:54.640713    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76" podStartSLOduration=1.6406944509999999 podStartE2EDuration="1.640694451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:13:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:13:54.638766622 +0000 UTC m=+927.930100440" watchObservedRunningTime="2026-09-14 13:13:54.640694451 +0000 UTC m=+927.932028269"
Sep 14 13:14:01.661870 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:01.661834    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:14:03.736911 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:03.736857    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:03.811322 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:03.811280    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:14:03.811499 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:03.811357    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:14:03.824194 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:03.824169    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:14:04.690900 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:04.690869    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:14:05.331005 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:05.330966    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:06.142400 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:06.142374    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:07.084859 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:07.084828    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:08.024377 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:08.024345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:08.992442 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:08.992414    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:09.827487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:09.827445    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:10.774820 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:10.774779    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:11.579090 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:11.579049    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:12.410993 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.410956    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"]
Sep 14 13:14:12.411451 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.411239    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf" podUID="4e1417a7-5c51-45af-8213-36db030d27a0" containerName="maas-api" containerID="cri-o://928904b12c864235e09accb0bdfc69762056b36586c09c80d8343a9028b280ee" gracePeriod=30
Sep 14 13:14:12.458098 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.458052    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:12.551821 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.551785    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:12.627190 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.627156    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:12.673318 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.673247    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:14:12.724560 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.724501    2581 generic.go:356] "Generic (PLEG): container finished" podID="4e1417a7-5c51-45af-8213-36db030d27a0" containerID="928904b12c864235e09accb0bdfc69762056b36586c09c80d8343a9028b280ee" exitCode=0
Sep 14 13:14:12.724763 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.724566    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf" event={"ID":"4e1417a7-5c51-45af-8213-36db030d27a0","Type":"ContainerDied","Data":"928904b12c864235e09accb0bdfc69762056b36586c09c80d8343a9028b280ee"}
Sep 14 13:14:12.724763 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.724623    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf" event={"ID":"4e1417a7-5c51-45af-8213-36db030d27a0","Type":"ContainerDied","Data":"3e4d5d4fa47ed9ffc2df126156be35fcd3fe451d8b6c7cfd6b647c0bd37bffc3"}
Sep 14 13:14:12.724763 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.724648    2581 scope.go:117] "RemoveContainer" containerID="928904b12c864235e09accb0bdfc69762056b36586c09c80d8343a9028b280ee"
Sep 14 13:14:12.724763 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.724577    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"
Sep 14 13:14:12.727083 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.727063    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e1417a7-5c51-45af-8213-36db030d27a0-openshift-service-ca\") pod \"4e1417a7-5c51-45af-8213-36db030d27a0\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") "
Sep 14 13:14:12.727196 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.727132    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qdcx\" (UniqueName: \"kubernetes.io/projected/4e1417a7-5c51-45af-8213-36db030d27a0-kube-api-access-9qdcx\") pod \"4e1417a7-5c51-45af-8213-36db030d27a0\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") "
Sep 14 13:14:12.727257 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.727193    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-metrics-certs\") pod \"4e1417a7-5c51-45af-8213-36db030d27a0\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") "
Sep 14 13:14:12.727314 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.727266    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-maas-api-tls\") pod \"4e1417a7-5c51-45af-8213-36db030d27a0\" (UID: \"4e1417a7-5c51-45af-8213-36db030d27a0\") "
Sep 14 13:14:12.727461 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.727440    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e1417a7-5c51-45af-8213-36db030d27a0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4e1417a7-5c51-45af-8213-36db030d27a0" (UID: "4e1417a7-5c51-45af-8213-36db030d27a0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:14:12.727674 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.727652    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e1417a7-5c51-45af-8213-36db030d27a0-openshift-service-ca\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:14:12.729375 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.729351    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4e1417a7-5c51-45af-8213-36db030d27a0" (UID: "4e1417a7-5c51-45af-8213-36db030d27a0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:14:12.729680 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.729655    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4e1417a7-5c51-45af-8213-36db030d27a0" (UID: "4e1417a7-5c51-45af-8213-36db030d27a0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:14:12.729765 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.729704    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e1417a7-5c51-45af-8213-36db030d27a0-kube-api-access-9qdcx" (OuterVolumeSpecName: "kube-api-access-9qdcx") pod "4e1417a7-5c51-45af-8213-36db030d27a0" (UID: "4e1417a7-5c51-45af-8213-36db030d27a0"). InnerVolumeSpecName "kube-api-access-9qdcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:14:12.734434 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.734408    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:12.748664 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.748643    2581 scope.go:117] "RemoveContainer" containerID="928904b12c864235e09accb0bdfc69762056b36586c09c80d8343a9028b280ee"
Sep 14 13:14:12.749003 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:14:12.748979    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"928904b12c864235e09accb0bdfc69762056b36586c09c80d8343a9028b280ee\": container with ID starting with 928904b12c864235e09accb0bdfc69762056b36586c09c80d8343a9028b280ee not found: ID does not exist" containerID="928904b12c864235e09accb0bdfc69762056b36586c09c80d8343a9028b280ee"
Sep 14 13:14:12.749104 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.749011    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"928904b12c864235e09accb0bdfc69762056b36586c09c80d8343a9028b280ee"} err="failed to get container status \"928904b12c864235e09accb0bdfc69762056b36586c09c80d8343a9028b280ee\": rpc error: code = NotFound desc = could not find container \"928904b12c864235e09accb0bdfc69762056b36586c09c80d8343a9028b280ee\": container with ID starting with 928904b12c864235e09accb0bdfc69762056b36586c09c80d8343a9028b280ee not found: ID does not exist"
Sep 14 13:14:12.828426 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.828387    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-metrics-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:14:12.828426 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.828427    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e1417a7-5c51-45af-8213-36db030d27a0-maas-api-tls\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:14:12.828657 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.828442    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qdcx\" (UniqueName: \"kubernetes.io/projected/4e1417a7-5c51-45af-8213-36db030d27a0-kube-api-access-9qdcx\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:14:12.855589 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:12.855561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:13.049741 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:13.049712    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"]
Sep 14 13:14:13.051521 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:13.051498    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dd15e28a-d45f645d5-j5jkf"]
Sep 14 13:14:13.237055 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:13.237021    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:13.246701 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:13.246666    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e1417a7-5c51-45af-8213-36db030d27a0" path="/var/lib/kubelet/pods/4e1417a7-5c51-45af-8213-36db030d27a0/volumes"
Sep 14 13:14:13.814001 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:13.813968    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:13.940838 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:13.940810    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:14.329552 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:14.329498    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:14.649384 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:14.649349    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:14.888326 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:14.888292    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:14.989579 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:14.989487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:15.561056 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:15.561023    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:16.353770 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:16.353737    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:16.972661 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:16.972621    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:17.114703 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:17.114672    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:17.367676 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:17.367642    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:18.259835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:18.259796    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:18.426599 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:18.426566    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:18.546777 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:18.546685    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:18.875379 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:18.875344    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:18.975284 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:18.975247    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:18.981896 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:18.981867    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"]
Sep 14 13:14:18.985247 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:18.983009    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e1417a7-5c51-45af-8213-36db030d27a0" containerName="maas-api"
Sep 14 13:14:18.985247 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:18.983038    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e1417a7-5c51-45af-8213-36db030d27a0" containerName="maas-api"
Sep 14 13:14:18.985247 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:18.983318    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e1417a7-5c51-45af-8213-36db030d27a0" containerName="maas-api"
Sep 14 13:14:18.989033 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:18.989003    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"
Sep 14 13:14:18.992082 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:18.992055    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-3448586b\""
Sep 14 13:14:19.000191 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.000162    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"]
Sep 14 13:14:19.092710 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.092670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsr68\" (UniqueName: \"kubernetes.io/projected/9fe14fc0-e95c-41d9-8c4c-6b588b4a9118-kube-api-access-rsr68\") pod \"payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69\" (UID: \"9fe14fc0-e95c-41d9-8c4c-6b588b4a9118\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"
Sep 14 13:14:19.092710 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.092720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fe14fc0-e95c-41d9-8c4c-6b588b4a9118-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69\" (UID: \"9fe14fc0-e95c-41d9-8c4c-6b588b4a9118\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"
Sep 14 13:14:19.166577 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.166474    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:19.194064 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.194029    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fe14fc0-e95c-41d9-8c4c-6b588b4a9118-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69\" (UID: \"9fe14fc0-e95c-41d9-8c4c-6b588b4a9118\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"
Sep 14 13:14:19.194194 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.194174    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsr68\" (UniqueName: \"kubernetes.io/projected/9fe14fc0-e95c-41d9-8c4c-6b588b4a9118-kube-api-access-rsr68\") pod \"payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69\" (UID: \"9fe14fc0-e95c-41d9-8c4c-6b588b4a9118\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"
Sep 14 13:14:19.194798 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.194776    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fe14fc0-e95c-41d9-8c4c-6b588b4a9118-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69\" (UID: \"9fe14fc0-e95c-41d9-8c4c-6b588b4a9118\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"
Sep 14 13:14:19.206485 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.206453    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsr68\" (UniqueName: \"kubernetes.io/projected/9fe14fc0-e95c-41d9-8c4c-6b588b4a9118-kube-api-access-rsr68\") pod \"payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69\" (UID: \"9fe14fc0-e95c-41d9-8c4c-6b588b4a9118\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"
Sep 14 13:14:19.267333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.267299    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:19.301443 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.301393    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"
Sep 14 13:14:19.375046 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.375013    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:19.578912 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.578887    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"]
Sep 14 13:14:19.580858 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:14:19.580831    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fe14fc0_e95c_41d9_8c4c_6b588b4a9118.slice/crio-6b03462f1139f861453834b9083b6706c33f593351583eda4fc1171b5d9a9e93 WatchSource:0}: Error finding container 6b03462f1139f861453834b9083b6706c33f593351583eda4fc1171b5d9a9e93: Status 404 returned error can't find the container with id 6b03462f1139f861453834b9083b6706c33f593351583eda4fc1171b5d9a9e93
Sep 14 13:14:19.763220 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.763184    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69" event={"ID":"9fe14fc0-e95c-41d9-8c4c-6b588b4a9118","Type":"ContainerStarted","Data":"9c1133f5528610dba3ca4109c3a72a8e98b22f96df26f4a378eff815c08aad2c"}
Sep 14 13:14:19.763444 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.763246    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69" event={"ID":"9fe14fc0-e95c-41d9-8c4c-6b588b4a9118","Type":"ContainerStarted","Data":"6b03462f1139f861453834b9083b6706c33f593351583eda4fc1171b5d9a9e93"}
Sep 14 13:14:19.763444 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.763316    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"
Sep 14 13:14:19.779794 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.779740    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69" podStartSLOduration=1.7797231 podStartE2EDuration="1.7797231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:14:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:14:19.778027179 +0000 UTC m=+953.069360996" watchObservedRunningTime="2026-09-14 13:14:19.7797231 +0000 UTC m=+953.071056961"
Sep 14 13:14:19.963290 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:19.963256    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:20.092479 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:20.092444    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:20.174631 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:20.174594    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:20.637474 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:20.637442    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:21.267358 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:21.267324    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:22.097716 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:22.097658    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:22.840279 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:22.840249    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:23.639211 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:23.639168    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:24.342175 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:24.342120    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:24.565556 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:24.562646    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:14:25.164991 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:25.164939    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:25.859232 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:25.859202    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:26.658877 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:26.658841    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:30.771925 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:30.771894    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"
Sep 14 13:14:31.959538 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:31.959495    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:32.508435 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.508382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:32.544987 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.544953    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"]
Sep 14 13:14:32.545220 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.545191    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns" podUID="05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e" containerName="payload-pre-processing" containerID="cri-o://397ad8301965d07224b8ff09d47b3253d79d236259b7a81967f3d0b19a92df6a" gracePeriod=30
Sep 14 13:14:32.616544 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.616499    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:32.744859 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.744820    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:32.816029 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.815932    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"
Sep 14 13:14:32.826691 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.826659    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:32.831699 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.831675    2581 generic.go:356] "Generic (PLEG): container finished" podID="05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e" containerID="397ad8301965d07224b8ff09d47b3253d79d236259b7a81967f3d0b19a92df6a" exitCode=0
Sep 14 13:14:32.831827 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.831719    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"
Sep 14 13:14:32.831827 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.831762    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns" event={"ID":"05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e","Type":"ContainerDied","Data":"397ad8301965d07224b8ff09d47b3253d79d236259b7a81967f3d0b19a92df6a"}
Sep 14 13:14:32.831827 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.831811    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns" event={"ID":"05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e","Type":"ContainerDied","Data":"e246ac9551d8cbf8a8fe75932ec564bf44f41660b87f2b22f92fd96321f4b07d"}
Sep 14 13:14:32.831935 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.831840    2581 scope.go:117] "RemoveContainer" containerID="397ad8301965d07224b8ff09d47b3253d79d236259b7a81967f3d0b19a92df6a"
Sep 14 13:14:32.843476 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.843455    2581 scope.go:117] "RemoveContainer" containerID="397ad8301965d07224b8ff09d47b3253d79d236259b7a81967f3d0b19a92df6a"
Sep 14 13:14:32.843840 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:14:32.843820    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"397ad8301965d07224b8ff09d47b3253d79d236259b7a81967f3d0b19a92df6a\": container with ID starting with 397ad8301965d07224b8ff09d47b3253d79d236259b7a81967f3d0b19a92df6a not found: ID does not exist" containerID="397ad8301965d07224b8ff09d47b3253d79d236259b7a81967f3d0b19a92df6a"
Sep 14 13:14:32.843923 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.843849    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"397ad8301965d07224b8ff09d47b3253d79d236259b7a81967f3d0b19a92df6a"} err="failed to get container status \"397ad8301965d07224b8ff09d47b3253d79d236259b7a81967f3d0b19a92df6a\": rpc error: code = NotFound desc = could not find container \"397ad8301965d07224b8ff09d47b3253d79d236259b7a81967f3d0b19a92df6a\": container with ID starting with 397ad8301965d07224b8ff09d47b3253d79d236259b7a81967f3d0b19a92df6a not found: ID does not exist"
Sep 14 13:14:32.936398 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.936362    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnw2c\" (UniqueName: \"kubernetes.io/projected/05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e-kube-api-access-nnw2c\") pod \"05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e\" (UID: \"05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e\") "
Sep 14 13:14:32.936625 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.936414    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e-plugins-config-volume\") pod \"05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e\" (UID: \"05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e\") "
Sep 14 13:14:32.936818 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.936794    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e" (UID: "05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:14:32.938495 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:32.938472    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e-kube-api-access-nnw2c" (OuterVolumeSpecName: "kube-api-access-nnw2c") pod "05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e" (UID: "05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e"). InnerVolumeSpecName "kube-api-access-nnw2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:14:33.037262 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:33.037221    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:14:33.037262 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:33.037254    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnw2c\" (UniqueName: \"kubernetes.io/projected/05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e-kube-api-access-nnw2c\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:14:33.153972 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:33.153940    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"]
Sep 14 13:14:33.156091 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:33.156061    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:33.156348 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:33.156330    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c14a5e-7c54bdfbb-9x5ns"]
Sep 14 13:14:33.246398 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:33.246358    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e" path="/var/lib/kubelet/pods/05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e/volumes"
Sep 14 13:14:33.257811 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:33.257787    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:33.963384 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:33.963351    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:34.723502 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:34.723466    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:35.434840 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:35.434805    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:36.356562 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:36.356520    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:37.272888 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:37.272851    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:38.058781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:38.058747    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:38.265955 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:38.265918    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:38.987099 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:38.987062    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:39.453869 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:39.453830    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:39.876813 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:39.876753    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:40.787106 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:40.787073    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:41.785698 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:41.785662    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:42.447080 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:42.447043    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:47.270509 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.270478    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:47.329498 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.329441    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"]
Sep 14 13:14:47.329756 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.329724    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69" podUID="9fe14fc0-e95c-41d9-8c4c-6b588b4a9118" containerName="payload-pre-processing" containerID="cri-o://9c1133f5528610dba3ca4109c3a72a8e98b22f96df26f4a378eff815c08aad2c" gracePeriod=30
Sep 14 13:14:47.364016 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.363979    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:47.464798 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.464762    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:47.563936 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.563847    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:47.598873 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.598843    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"
Sep 14 13:14:47.689581 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.689548    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsr68\" (UniqueName: \"kubernetes.io/projected/9fe14fc0-e95c-41d9-8c4c-6b588b4a9118-kube-api-access-rsr68\") pod \"9fe14fc0-e95c-41d9-8c4c-6b588b4a9118\" (UID: \"9fe14fc0-e95c-41d9-8c4c-6b588b4a9118\") "
Sep 14 13:14:47.689767 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.689599    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fe14fc0-e95c-41d9-8c4c-6b588b4a9118-plugins-config-volume\") pod \"9fe14fc0-e95c-41d9-8c4c-6b588b4a9118\" (UID: \"9fe14fc0-e95c-41d9-8c4c-6b588b4a9118\") "
Sep 14 13:14:47.690018 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.689992    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fe14fc0-e95c-41d9-8c4c-6b588b4a9118-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9fe14fc0-e95c-41d9-8c4c-6b588b4a9118" (UID: "9fe14fc0-e95c-41d9-8c4c-6b588b4a9118"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:14:47.691635 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.691604    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fe14fc0-e95c-41d9-8c4c-6b588b4a9118-kube-api-access-rsr68" (OuterVolumeSpecName: "kube-api-access-rsr68") pod "9fe14fc0-e95c-41d9-8c4c-6b588b4a9118" (UID: "9fe14fc0-e95c-41d9-8c4c-6b588b4a9118"). InnerVolumeSpecName "kube-api-access-rsr68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:14:47.695632 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.695606    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:47.790486 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.790453    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsr68\" (UniqueName: \"kubernetes.io/projected/9fe14fc0-e95c-41d9-8c4c-6b588b4a9118-kube-api-access-rsr68\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:14:47.790486 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.790480    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fe14fc0-e95c-41d9-8c4c-6b588b4a9118-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:14:47.909996 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.909964    2581 generic.go:356] "Generic (PLEG): container finished" podID="9fe14fc0-e95c-41d9-8c4c-6b588b4a9118" containerID="9c1133f5528610dba3ca4109c3a72a8e98b22f96df26f4a378eff815c08aad2c" exitCode=0
Sep 14 13:14:47.910204 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.910017    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"
Sep 14 13:14:47.910204 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.910049    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69" event={"ID":"9fe14fc0-e95c-41d9-8c4c-6b588b4a9118","Type":"ContainerDied","Data":"9c1133f5528610dba3ca4109c3a72a8e98b22f96df26f4a378eff815c08aad2c"}
Sep 14 13:14:47.910204 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.910112    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69" event={"ID":"9fe14fc0-e95c-41d9-8c4c-6b588b4a9118","Type":"ContainerDied","Data":"6b03462f1139f861453834b9083b6706c33f593351583eda4fc1171b5d9a9e93"}
Sep 14 13:14:47.910204 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.910136    2581 scope.go:117] "RemoveContainer" containerID="9c1133f5528610dba3ca4109c3a72a8e98b22f96df26f4a378eff815c08aad2c"
Sep 14 13:14:47.919613 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.919581    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:47.920000 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.919984    2581 scope.go:117] "RemoveContainer" containerID="9c1133f5528610dba3ca4109c3a72a8e98b22f96df26f4a378eff815c08aad2c"
Sep 14 13:14:47.920271 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:14:47.920251    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c1133f5528610dba3ca4109c3a72a8e98b22f96df26f4a378eff815c08aad2c\": container with ID starting with 9c1133f5528610dba3ca4109c3a72a8e98b22f96df26f4a378eff815c08aad2c not found: ID does not exist" containerID="9c1133f5528610dba3ca4109c3a72a8e98b22f96df26f4a378eff815c08aad2c"
Sep 14 13:14:47.920318 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.920280    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c1133f5528610dba3ca4109c3a72a8e98b22f96df26f4a378eff815c08aad2c"} err="failed to get container status \"9c1133f5528610dba3ca4109c3a72a8e98b22f96df26f4a378eff815c08aad2c\": rpc error: code = NotFound desc = could not find container \"9c1133f5528610dba3ca4109c3a72a8e98b22f96df26f4a378eff815c08aad2c\": container with ID starting with 9c1133f5528610dba3ca4109c3a72a8e98b22f96df26f4a378eff815c08aad2c not found: ID does not exist"
Sep 14 13:14:47.933275 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.933248    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"]
Sep 14 13:14:47.938095 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:47.938058    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3448586b-7c9c885c74-scz69"]
Sep 14 13:14:48.335650 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:48.335562    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:48.423403 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:48.423372    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:48.554357 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:48.554316    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:48.756872 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:48.756837    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:49.245981 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:49.245948    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fe14fc0-e95c-41d9-8c4c-6b588b4a9118" path="/var/lib/kubelet/pods/9fe14fc0-e95c-41d9-8c4c-6b588b4a9118/volumes"
Sep 14 13:14:49.275941 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:49.275907    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:14:49.666594 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:49.666561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:49.910498 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:49.910456    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:50.533190 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:50.533143    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:51.239429 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:51.239396    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:56.138346 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:56.138312    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:56.242784 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:56.242740    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:56.341199 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:56.341164    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:56.439724 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:56.439634    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:56.821403 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:56.821306    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:57.085473 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:14:57.085380    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:00.162368 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.162336    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:00.206551 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.206492    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29823195-65drz"]
Sep 14 13:15:00.206948 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.206933    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e" containerName="payload-pre-processing"
Sep 14 13:15:00.206990 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.206950    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e" containerName="payload-pre-processing"
Sep 14 13:15:00.206990 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.206958    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fe14fc0-e95c-41d9-8c4c-6b588b4a9118" containerName="payload-pre-processing"
Sep 14 13:15:00.206990 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.206964    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe14fc0-e95c-41d9-8c4c-6b588b4a9118" containerName="payload-pre-processing"
Sep 14 13:15:00.207121 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.207055    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fe14fc0-e95c-41d9-8c4c-6b588b4a9118" containerName="payload-pre-processing"
Sep 14 13:15:00.207121 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.207064    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="05cd1fd5-f01c-47ec-b8a6-f18c014d0a2e" containerName="payload-pre-processing"
Sep 14 13:15:00.210202 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.210178    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29823195-65drz"
Sep 14 13:15:00.210811 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.210787    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b3d6c5-29823195-j6brt"]
Sep 14 13:15:00.212926 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.212904    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbbzp\""
Sep 14 13:15:00.213961 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.213942    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b3d6c5-29823195-j6brt"
Sep 14 13:15:00.223607 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.223581    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29823195-65drz"
Sep 14 13:15:00.223780 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.223658    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29823195-65drz"]
Sep 14 13:15:00.226701 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.226269    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b3d6c5-29823195-j6brt"]
Sep 14 13:15:00.233732 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.233711    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b3d6c5-29823195-j6brt"
Sep 14 13:15:00.505404 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.505377    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29823195-65drz"]
Sep 14 13:15:00.508269 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:15:00.508240    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd18b195_fbf1_4ed8_9dd3_c5c5f68a6e07.slice/crio-5a310c265a6cafbc2fa2642f4613f19b00cefe952e68780363dfeda633e0fc66 WatchSource:0}: Error finding container 5a310c265a6cafbc2fa2642f4613f19b00cefe952e68780363dfeda633e0fc66: Status 404 returned error can't find the container with id 5a310c265a6cafbc2fa2642f4613f19b00cefe952e68780363dfeda633e0fc66
Sep 14 13:15:00.525208 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.525179    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b3d6c5-29823195-j6brt"]
Sep 14 13:15:00.526183 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:15:00.526155    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4aae587_eeea_4aa8_a45a_4255945c1ede.slice/crio-cddfe4d8e220206211fb2c9bcad8c6ac5caab6b0f556dfeebc04bef1fc307da5 WatchSource:0}: Error finding container cddfe4d8e220206211fb2c9bcad8c6ac5caab6b0f556dfeebc04bef1fc307da5: Status 404 returned error can't find the container with id cddfe4d8e220206211fb2c9bcad8c6ac5caab6b0f556dfeebc04bef1fc307da5
Sep 14 13:15:00.980005 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.979971    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29823195-65drz" event={"ID":"cd18b195-fbf1-4ed8-9dd3-c5c5f68a6e07","Type":"ContainerStarted","Data":"5a310c265a6cafbc2fa2642f4613f19b00cefe952e68780363dfeda633e0fc66"}
Sep 14 13:15:00.981241 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:00.981217    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b3d6c5-29823195-j6brt" event={"ID":"a4aae587-eeea-4aa8-a45a-4255945c1ede","Type":"ContainerStarted","Data":"cddfe4d8e220206211fb2c9bcad8c6ac5caab6b0f556dfeebc04bef1fc307da5"}
Sep 14 13:15:04.001917 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:04.001880    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b3d6c5-29823195-j6brt" event={"ID":"a4aae587-eeea-4aa8-a45a-4255945c1ede","Type":"ContainerStarted","Data":"a4145e2cfec6fcebdc794ba9713ffdfe5727d5f4658bc080e44e1464825f55d8"}
Sep 14 13:15:04.004431 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:04.004401    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29823195-65drz" event={"ID":"cd18b195-fbf1-4ed8-9dd3-c5c5f68a6e07","Type":"ContainerStarted","Data":"cac9df93ce6b52e0957e35f34737a2dfd239c5c3844c1a00c5260b2b525c5f2b"}
Sep 14 13:15:04.017976 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:04.017930    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b3d6c5-29823195-j6brt" podStartSLOduration=1.551338908 podStartE2EDuration="4.017911551s" totalImagesPullingTime="2.466572643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:15:04.015640501 +0000 UTC m=+997.306974318" watchObservedRunningTime="2026-09-14 13:15:04.017911551 +0000 UTC m=+997.309245370"
Sep 14 13:15:04.035204 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:04.035156    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29823195-65drz" podStartSLOduration=1.5451820870000001 podStartE2EDuration="4.035139622s" totalImagesPullingTime="2.489957535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:15:04.033989057 +0000 UTC m=+997.325322875" watchObservedRunningTime="2026-09-14 13:15:04.035139622 +0000 UTC m=+997.326473450"
Sep 14 13:15:05.014452 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:05.014409    2581 generic.go:356] "Generic (PLEG): container finished" podID="cd18b195-fbf1-4ed8-9dd3-c5c5f68a6e07" containerID="cac9df93ce6b52e0957e35f34737a2dfd239c5c3844c1a00c5260b2b525c5f2b" exitCode=0
Sep 14 13:15:05.014931 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:05.014498    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29823195-65drz" event={"ID":"cd18b195-fbf1-4ed8-9dd3-c5c5f68a6e07","Type":"ContainerDied","Data":"cac9df93ce6b52e0957e35f34737a2dfd239c5c3844c1a00c5260b2b525c5f2b"}
Sep 14 13:15:05.016175 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:05.016144    2581 generic.go:356] "Generic (PLEG): container finished" podID="a4aae587-eeea-4aa8-a45a-4255945c1ede" containerID="a4145e2cfec6fcebdc794ba9713ffdfe5727d5f4658bc080e44e1464825f55d8" exitCode=0
Sep 14 13:15:05.016299 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:05.016178    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b3d6c5-29823195-j6brt" event={"ID":"a4aae587-eeea-4aa8-a45a-4255945c1ede","Type":"ContainerDied","Data":"a4145e2cfec6fcebdc794ba9713ffdfe5727d5f4658bc080e44e1464825f55d8"}
Sep 14 13:15:05.558084 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:05.558045    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:15:06.149332 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:06.149304    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b3d6c5-29823195-j6brt"
Sep 14 13:15:06.180547 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:06.180511    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29823195-65drz"
Sep 14 13:15:07.028210 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:07.028180    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b3d6c5-29823195-j6brt"
Sep 14 13:15:07.028210 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:07.028188    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b3d6c5-29823195-j6brt" event={"ID":"a4aae587-eeea-4aa8-a45a-4255945c1ede","Type":"ContainerDied","Data":"cddfe4d8e220206211fb2c9bcad8c6ac5caab6b0f556dfeebc04bef1fc307da5"}
Sep 14 13:15:07.028455 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:07.028238    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cddfe4d8e220206211fb2c9bcad8c6ac5caab6b0f556dfeebc04bef1fc307da5"
Sep 14 13:15:07.029471 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:07.029450    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29823195-65drz"
Sep 14 13:15:07.029602 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:07.029470    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29823195-65drz" event={"ID":"cd18b195-fbf1-4ed8-9dd3-c5c5f68a6e07","Type":"ContainerDied","Data":"5a310c265a6cafbc2fa2642f4613f19b00cefe952e68780363dfeda633e0fc66"}
Sep 14 13:15:07.029602 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:07.029507    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a310c265a6cafbc2fa2642f4613f19b00cefe952e68780363dfeda633e0fc66"
Sep 14 13:15:15.846290 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:15.846251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:16.117005 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:16.116971    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:16.534121 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:16.534016    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:16.853942 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:16.853904    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:17.106304 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:17.106216    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:19.648778 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:19.648735    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:20.324171 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:20.324134    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:20.416946 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:20.416906    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:21.454720 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:21.454687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:22.219948 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:22.219912    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:22.755525 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:22.755490    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:22.959952 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:22.959904    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:23.699492 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:23.699455    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:23.800379 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:23.800328    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:24.153462 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.153428    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:24.257266 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.257228    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:24.359430 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.359381    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:24.401243 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.401198    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"]
Sep 14 13:15:24.401663 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.401647    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4aae587-eeea-4aa8-a45a-4255945c1ede" containerName="cleanup"
Sep 14 13:15:24.401728 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.401667    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4aae587-eeea-4aa8-a45a-4255945c1ede" containerName="cleanup"
Sep 14 13:15:24.401728 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.401685    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd18b195-fbf1-4ed8-9dd3-c5c5f68a6e07" containerName="cleanup"
Sep 14 13:15:24.401728 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.401691    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd18b195-fbf1-4ed8-9dd3-c5c5f68a6e07" containerName="cleanup"
Sep 14 13:15:24.401830 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.401750    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4aae587-eeea-4aa8-a45a-4255945c1ede" containerName="cleanup"
Sep 14 13:15:24.401830 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.401759    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd18b195-fbf1-4ed8-9dd3-c5c5f68a6e07" containerName="cleanup"
Sep 14 13:15:24.405027 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.404971    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"
Sep 14 13:15:24.407709 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.407678    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-ace1c6-dockercfg-b5kgk\""
Sep 14 13:15:24.407709 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.407704    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-ace1c6\""
Sep 14 13:15:24.415997 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.415972    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"]
Sep 14 13:15:24.434464 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.434428    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scvtv\" (UniqueName: \"kubernetes.io/projected/69e8444f-e219-4301-8491-8ee0c412ed6f-kube-api-access-scvtv\") pod \"payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln\" (UID: \"69e8444f-e219-4301-8491-8ee0c412ed6f\") " pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"
Sep 14 13:15:24.434656 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.434484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69e8444f-e219-4301-8491-8ee0c412ed6f-plugins-config-volume\") pod \"payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln\" (UID: \"69e8444f-e219-4301-8491-8ee0c412ed6f\") " pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"
Sep 14 13:15:24.535928 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.535890    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scvtv\" (UniqueName: \"kubernetes.io/projected/69e8444f-e219-4301-8491-8ee0c412ed6f-kube-api-access-scvtv\") pod \"payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln\" (UID: \"69e8444f-e219-4301-8491-8ee0c412ed6f\") " pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"
Sep 14 13:15:24.536146 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.535941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69e8444f-e219-4301-8491-8ee0c412ed6f-plugins-config-volume\") pod \"payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln\" (UID: \"69e8444f-e219-4301-8491-8ee0c412ed6f\") " pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"
Sep 14 13:15:24.536770 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.536742    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69e8444f-e219-4301-8491-8ee0c412ed6f-plugins-config-volume\") pod \"payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln\" (UID: \"69e8444f-e219-4301-8491-8ee0c412ed6f\") " pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"
Sep 14 13:15:24.546960 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.546936    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scvtv\" (UniqueName: \"kubernetes.io/projected/69e8444f-e219-4301-8491-8ee0c412ed6f-kube-api-access-scvtv\") pod \"payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln\" (UID: \"69e8444f-e219-4301-8491-8ee0c412ed6f\") " pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"
Sep 14 13:15:24.716996 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.716903    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"
Sep 14 13:15:24.996554 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:24.996509    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"]
Sep 14 13:15:24.998593 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:15:24.998554    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69e8444f_e219_4301_8491_8ee0c412ed6f.slice/crio-4a467205d41c21cb89a2fc50e5cb4923764d920efa14f873233319744084c9ee WatchSource:0}: Error finding container 4a467205d41c21cb89a2fc50e5cb4923764d920efa14f873233319744084c9ee: Status 404 returned error can't find the container with id 4a467205d41c21cb89a2fc50e5cb4923764d920efa14f873233319744084c9ee
Sep 14 13:15:25.138495 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:25.138460    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln" event={"ID":"69e8444f-e219-4301-8491-8ee0c412ed6f","Type":"ContainerStarted","Data":"971710f565a0b9905c2cbdcae814ad3f2f88ef5f563b3ace4aa9ad0e5bf78b07"}
Sep 14 13:15:25.138701 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:25.138512    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln" event={"ID":"69e8444f-e219-4301-8491-8ee0c412ed6f","Type":"ContainerStarted","Data":"4a467205d41c21cb89a2fc50e5cb4923764d920efa14f873233319744084c9ee"}
Sep 14 13:15:25.138701 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:25.138580    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"
Sep 14 13:15:25.157644 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:25.157593    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln" podStartSLOduration=1.157576881 podStartE2EDuration="1.157576881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:15:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:15:25.154919332 +0000 UTC m=+1018.446253148" watchObservedRunningTime="2026-09-14 13:15:25.157576881 +0000 UTC m=+1018.448910698"
Sep 14 13:15:28.694166 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:28.694134    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:28.809466 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:28.809424    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:28.901742 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:28.901680    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:33.258905 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:33.258871    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:15:34.138914 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:34.138876    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:36.145686 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:36.145654    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"
Sep 14 13:15:44.794835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:44.794784    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:46.412911 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:46.412877    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:47.586203 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:47.586170    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:48.752313 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:48.752273    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:49.529871 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:49.529837    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:50.389123 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:50.389090    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:50.698483 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:50.698383    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:51.068189 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:51.068095    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:15:51.582977 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:51.582944    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:52.433034 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:52.432997    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:52.535407 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:52.535373    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:52.634462 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:15:52.634423    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:03.178045 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:03.178009    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:03.887465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:03.887428    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:06.163705 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:06.163671    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:16:09.919749 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:09.919714    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:10.020789 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:10.020757    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:10.312814 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:10.312725    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:10.422667 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:10.422632    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:10.515778 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:10.515746    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:10.694319 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:10.694287    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:11.286576 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:11.286522    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:11.385595 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:11.385558    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:12.992004 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:12.991967    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:20.037970 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:20.037883    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:20.138725 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:20.138687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:20.712410 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:20.712374    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:26.818672 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:26.818632    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:26.878090 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:26.878050    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:27.036597 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:27.036554    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:27.349937 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:27.349901    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:27.508050 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:27.508016    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:31.752892 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:31.752856    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:16:32.331302 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:32.331267    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:33.136009 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:33.135974    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:33.236603 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:33.236573    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:34.112466 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:34.112432    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:34.840311 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:34.840273    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:35.709919 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:35.709884    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:36.337803 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:36.337770    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:36.961650 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:36.961619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:37.020821 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:37.020785    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:37.122266 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:37.122227    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:37.473882 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:37.473843    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:37.551777 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:37.551743    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:46.876285 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:46.876251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:58.862298 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:58.862204    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:59.544251 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:16:59.544207    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:00.386832 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:00.386801    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:01.044361 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:01.044330    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:01.926266 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:01.926221    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:02.658651 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:02.658616    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:17:02.859295 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:02.859259    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:03.071880 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.071801    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:03.330212 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.330123    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"]
Sep 14 13:17:03.333460 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.333436    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.338168 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.338141    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-ace1c6\"/\"default-dockercfg-5cvnc\""
Sep 14 13:17:03.338323 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.338141    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-ace1c6\"/\"openshift-service-ca.crt\""
Sep 14 13:17:03.338323 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.338197    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-ace1c6\"/\"kube-root-ca.crt\""
Sep 14 13:17:03.338323 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.338160    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-ace1c6\"/\"explicit-model-ace1c6-kserve-self-signed-certs\""
Sep 14 13:17:03.345419 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.345386    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"]
Sep 14 13:17:03.364904 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.364867    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:03.500012 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.499981    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a5b773d-3a45-42e5-905c-7794efe7560e-tls-certs\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.500178 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.500032    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-tmp-dir\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.500178 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.500108    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-model-cache\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.500178 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.500164    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-dshm\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.500307 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.500188    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46l42\" (UniqueName: \"kubernetes.io/projected/9a5b773d-3a45-42e5-905c-7794efe7560e-kube-api-access-46l42\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.500307 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.500251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-home\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.601715 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.601639    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-model-cache\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.601892 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.601723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-dshm\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.601892 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.601749    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46l42\" (UniqueName: \"kubernetes.io/projected/9a5b773d-3a45-42e5-905c-7794efe7560e-kube-api-access-46l42\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.601892 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.601783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-home\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.601892 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.601801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a5b773d-3a45-42e5-905c-7794efe7560e-tls-certs\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.601892 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.601840    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-tmp-dir\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.602165 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.602137    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-model-cache\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.602297 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.602272    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-tmp-dir\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.602297 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.602290    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-home\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.604075 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.604051    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-dshm\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.604487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.604469    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a5b773d-3a45-42e5-905c-7794efe7560e-tls-certs\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.609782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.609759    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46l42\" (UniqueName: \"kubernetes.io/projected/9a5b773d-3a45-42e5-905c-7794efe7560e-kube-api-access-46l42\") pod \"explicit-model-ace1c6-kserve-5f9f987b75-jbcsq\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") " pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.643854 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.643815    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:03.704070 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.703979    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:03.810434 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.810403    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:03.923094 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:03.923070    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"]
Sep 14 13:17:03.924972 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:17:03.924939    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a5b773d_3a45_42e5_905c_7794efe7560e.slice/crio-c857a40d1289f71286bc356f3ff534c37f939e1670f219592040db992a432c7e WatchSource:0}: Error finding container c857a40d1289f71286bc356f3ff534c37f939e1670f219592040db992a432c7e: Status 404 returned error can't find the container with id c857a40d1289f71286bc356f3ff534c37f939e1670f219592040db992a432c7e
Sep 14 13:17:04.124625 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:04.124581    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:04.682725 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:04.682691    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq" event={"ID":"9a5b773d-3a45-42e5-905c-7794efe7560e","Type":"ContainerStarted","Data":"6c8363157074aa2c785db3121f36fdcd9ce42b0ee6aee90c057eb3e83efcd5fa"}
Sep 14 13:17:04.682912 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:04.682737    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq" event={"ID":"9a5b773d-3a45-42e5-905c-7794efe7560e","Type":"ContainerStarted","Data":"c857a40d1289f71286bc356f3ff534c37f939e1670f219592040db992a432c7e"}
Sep 14 13:17:04.707015 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:04.706967    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq" podStartSLOduration=1.706949765 podStartE2EDuration="1.706949765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:17:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:17:04.703545739 +0000 UTC m=+1117.994879568" watchObservedRunningTime="2026-09-14 13:17:04.706949765 +0000 UTC m=+1117.998283628"
Sep 14 13:17:10.334955 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:10.334919    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:10.397875 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:10.397842    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:11.539017 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:11.538980    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:12.191910 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:12.191871    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:12.877517 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:12.877484    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:13.644510 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:13.644473    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:13.644724 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:13.644544    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:13.657425 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:13.657383    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:13.735898 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:13.735862    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:13.742421 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:13.742394    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:17:14.346263 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:14.346229    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:14.962212 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:14.962155    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:15.644489 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:15.644458    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:21.169211 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:21.169158    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:21.240667 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:21.240633    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:21.317985 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:21.317954    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:21.390245 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:21.390207    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:21.787412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:21.787382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:21.892316 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:21.892284    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:22.753953 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:22.753910    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:22.855636 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:22.855602    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:17:23.421100 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:23.421066    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:24.173690 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:24.173656    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:24.894820 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:24.894785    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:25.515588 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:25.515553    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:27.938507 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:27.938471    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:28.255880 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:28.255799    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:28.321724 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:28.321687    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:28.526906 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:28.526830    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:28.858332 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:28.858297    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:28.959912 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:28.959881    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:29.084650 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:29.084617    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:35.298722 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:35.298673    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:35.407415 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:35.407380    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:35.499736 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:35.499680    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:35.635305 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:35.635275    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:38.365963 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:38.365935    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:39.111378 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:39.111344    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:39.930597 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:39.930563    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:40.536312 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:40.536277    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:41.221268 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:41.221211    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:41.822544 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:41.822496    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:41.923480 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:41.923447    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:42.308664 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.308624    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"]
Sep 14 13:17:42.312743 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.312719    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.316259 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.316229    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-bc851d\"/\"distinct-w0-bc851d-kserve-self-signed-certs\""
Sep 14 13:17:42.323164 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.323136    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"]
Sep 14 13:17:42.328125 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.328099    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:42.393459 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.393424    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-tmp-dir\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.393676 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.393470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-dshm\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.393676 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.393490    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-model-cache\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.393676 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.393574    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-tls-certs\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.393676 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.393610    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-home\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.393676 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.393669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7d22\" (UniqueName: \"kubernetes.io/projected/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-kube-api-access-l7d22\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.494868 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.494830    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7d22\" (UniqueName: \"kubernetes.io/projected/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-kube-api-access-l7d22\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.495033 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.494886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-tmp-dir\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.495033 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.494924    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-dshm\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.495033 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.494939    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-model-cache\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.495033 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.494970    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-tls-certs\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.495033 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.494988    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-home\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.495419 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.495398    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-model-cache\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.495488 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.495456    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-tmp-dir\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.495557 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.495498    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-home\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.497319 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.497295    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-dshm\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.497646 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.497612    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-tls-certs\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.502603 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.502583    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7d22\" (UniqueName: \"kubernetes.io/projected/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-kube-api-access-l7d22\") pod \"distinct-w0-bc851d-kserve-6b59df5794-2xn9x\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.623997 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.623951    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:42.702714 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.702677    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:42.801928 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.801889    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:42.897846 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:42.897653    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"]
Sep 14 13:17:42.900881 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:17:42.900849    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c7d4e50_3882_439d_8202_bb7efdb6e8e9.slice/crio-4a328a1f554157cef791b3d475de1f5ed4142c5bcef4acd5ec1a76a231f33ba7 WatchSource:0}: Error finding container 4a328a1f554157cef791b3d475de1f5ed4142c5bcef4acd5ec1a76a231f33ba7: Status 404 returned error can't find the container with id 4a328a1f554157cef791b3d475de1f5ed4142c5bcef4acd5ec1a76a231f33ba7
Sep 14 13:17:43.031814 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:43.031772    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:43.152862 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:43.152771    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:43.896879 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:43.896840    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x" event={"ID":"5c7d4e50-3882-439d-8202-bb7efdb6e8e9","Type":"ContainerStarted","Data":"1e03508d655a6539ade48144730ebe6e8b7ac1662f08c1413aac8a8b37697261"}
Sep 14 13:17:43.896879 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:43.896883    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x" event={"ID":"5c7d4e50-3882-439d-8202-bb7efdb6e8e9","Type":"ContainerStarted","Data":"4a328a1f554157cef791b3d475de1f5ed4142c5bcef4acd5ec1a76a231f33ba7"}
Sep 14 13:17:43.914647 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:43.914579    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x" podStartSLOduration=1.914560601 podStartE2EDuration="1.914560601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:17:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:17:43.913690518 +0000 UTC m=+1157.205024338" watchObservedRunningTime="2026-09-14 13:17:43.914560601 +0000 UTC m=+1157.205894418"
Sep 14 13:17:45.504195 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:45.504156    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"]
Sep 14 13:17:45.511090 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:45.511056    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:45.611458 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:45.611425    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:46.175942 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:46.175892    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:46.272172 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:46.272129    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:46.522454 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:46.522371    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:46.900043 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:46.900011    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"]
Sep 14 13:17:46.928773 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:46.928736    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:47.260246 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:47.260162    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:48.047015 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.046987    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:48.612112 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.612084    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:48.636292 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.636254    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"]
Sep 14 13:17:48.636550 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.636508    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln" podUID="69e8444f-e219-4301-8491-8ee0c412ed6f" containerName="payload-processing" containerID="cri-o://971710f565a0b9905c2cbdcae814ad3f2f88ef5f563b3ace4aa9ad0e5bf78b07" gracePeriod=30
Sep 14 13:17:48.710012 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.709974    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:48.825079 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.825047    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:48.893232 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.893207    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:48.903996 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.903970    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"
Sep 14 13:17:48.926793 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.926762    2581 generic.go:356] "Generic (PLEG): container finished" podID="69e8444f-e219-4301-8491-8ee0c412ed6f" containerID="971710f565a0b9905c2cbdcae814ad3f2f88ef5f563b3ace4aa9ad0e5bf78b07" exitCode=0
Sep 14 13:17:48.926995 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.926801    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"
Sep 14 13:17:48.927081 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.926836    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln" event={"ID":"69e8444f-e219-4301-8491-8ee0c412ed6f","Type":"ContainerDied","Data":"971710f565a0b9905c2cbdcae814ad3f2f88ef5f563b3ace4aa9ad0e5bf78b07"}
Sep 14 13:17:48.927167 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.927145    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln" event={"ID":"69e8444f-e219-4301-8491-8ee0c412ed6f","Type":"ContainerDied","Data":"4a467205d41c21cb89a2fc50e5cb4923764d920efa14f873233319744084c9ee"}
Sep 14 13:17:48.927997 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.927189    2581 scope.go:117] "RemoveContainer" containerID="971710f565a0b9905c2cbdcae814ad3f2f88ef5f563b3ace4aa9ad0e5bf78b07"
Sep 14 13:17:48.937958 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.937933    2581 scope.go:117] "RemoveContainer" containerID="971710f565a0b9905c2cbdcae814ad3f2f88ef5f563b3ace4aa9ad0e5bf78b07"
Sep 14 13:17:48.938272 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:17:48.938244    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"971710f565a0b9905c2cbdcae814ad3f2f88ef5f563b3ace4aa9ad0e5bf78b07\": container with ID starting with 971710f565a0b9905c2cbdcae814ad3f2f88ef5f563b3ace4aa9ad0e5bf78b07 not found: ID does not exist" containerID="971710f565a0b9905c2cbdcae814ad3f2f88ef5f563b3ace4aa9ad0e5bf78b07"
Sep 14 13:17:48.938333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.938286    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"971710f565a0b9905c2cbdcae814ad3f2f88ef5f563b3ace4aa9ad0e5bf78b07"} err="failed to get container status \"971710f565a0b9905c2cbdcae814ad3f2f88ef5f563b3ace4aa9ad0e5bf78b07\": rpc error: code = NotFound desc = could not find container \"971710f565a0b9905c2cbdcae814ad3f2f88ef5f563b3ace4aa9ad0e5bf78b07\": container with ID starting with 971710f565a0b9905c2cbdcae814ad3f2f88ef5f563b3ace4aa9ad0e5bf78b07 not found: ID does not exist"
Sep 14 13:17:48.960096 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.960070    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69e8444f-e219-4301-8491-8ee0c412ed6f-plugins-config-volume\") pod \"69e8444f-e219-4301-8491-8ee0c412ed6f\" (UID: \"69e8444f-e219-4301-8491-8ee0c412ed6f\") "
Sep 14 13:17:48.960227 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.960171    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scvtv\" (UniqueName: \"kubernetes.io/projected/69e8444f-e219-4301-8491-8ee0c412ed6f-kube-api-access-scvtv\") pod \"69e8444f-e219-4301-8491-8ee0c412ed6f\" (UID: \"69e8444f-e219-4301-8491-8ee0c412ed6f\") "
Sep 14 13:17:48.960580 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.960511    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69e8444f-e219-4301-8491-8ee0c412ed6f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69e8444f-e219-4301-8491-8ee0c412ed6f" (UID: "69e8444f-e219-4301-8491-8ee0c412ed6f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:17:48.962272 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:48.962245    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69e8444f-e219-4301-8491-8ee0c412ed6f-kube-api-access-scvtv" (OuterVolumeSpecName: "kube-api-access-scvtv") pod "69e8444f-e219-4301-8491-8ee0c412ed6f" (UID: "69e8444f-e219-4301-8491-8ee0c412ed6f"). InnerVolumeSpecName "kube-api-access-scvtv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:17:49.061857 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:49.061823    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69e8444f-e219-4301-8491-8ee0c412ed6f-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:17:49.061857 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:49.061852    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scvtv\" (UniqueName: \"kubernetes.io/projected/69e8444f-e219-4301-8491-8ee0c412ed6f-kube-api-access-scvtv\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:17:49.169218 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:49.169125    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:49.250658 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:49.250620    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"]
Sep 14 13:17:49.252494 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:49.252471    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-ace1c6-5f5665f86f-xsxln"]
Sep 14 13:17:49.572054 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:49.571938    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:49.633841 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:49.633808    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:49.872772 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:49.872731    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:50.170061 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:50.169984    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:50.258273 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:50.258237    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:50.626985 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:50.626952    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:51.251336 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:51.251294    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69e8444f-e219-4301-8491-8ee0c412ed6f" path="/var/lib/kubelet/pods/69e8444f-e219-4301-8491-8ee0c412ed6f/volumes"
Sep 14 13:17:51.339697 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:51.339661    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:51.957690 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:51.957651    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:52.624576 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:52.624520    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:52.625072 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:52.624598    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:52.637372 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:52.637345    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:52.651101 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:52.651077    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:52.960599 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:52.960511    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:17:53.418729 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:53.418695    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:54.149740 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:54.149699    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:54.351027 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:54.350994    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:54.963228 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:17:54.963194    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:18:00.524924 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.524857    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq" podUID="9a5b773d-3a45-42e5-905c-7794efe7560e" containerName="main" containerID="cri-o://6c8363157074aa2c785db3121f36fdcd9ce42b0ee6aee90c057eb3e83efcd5fa" gracePeriod=45
Sep 14 13:18:00.569345 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:00.569309    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a5b773d_3a45_42e5_905c_7794efe7560e.slice/crio-6c8363157074aa2c785db3121f36fdcd9ce42b0ee6aee90c057eb3e83efcd5fa.scope\": RecentStats: unable to find data in memory cache]"
Sep 14 13:18:00.776433 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.776365    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:18:00.880772 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.880735    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-model-cache\") pod \"9a5b773d-3a45-42e5-905c-7794efe7560e\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") "
Sep 14 13:18:00.880966 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.880813    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46l42\" (UniqueName: \"kubernetes.io/projected/9a5b773d-3a45-42e5-905c-7794efe7560e-kube-api-access-46l42\") pod \"9a5b773d-3a45-42e5-905c-7794efe7560e\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") "
Sep 14 13:18:00.880966 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.880860    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-dshm\") pod \"9a5b773d-3a45-42e5-905c-7794efe7560e\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") "
Sep 14 13:18:00.880966 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.880905    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a5b773d-3a45-42e5-905c-7794efe7560e-tls-certs\") pod \"9a5b773d-3a45-42e5-905c-7794efe7560e\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") "
Sep 14 13:18:00.880966 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.880927    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-home\") pod \"9a5b773d-3a45-42e5-905c-7794efe7560e\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") "
Sep 14 13:18:00.881180 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.880971    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-tmp-dir\") pod \"9a5b773d-3a45-42e5-905c-7794efe7560e\" (UID: \"9a5b773d-3a45-42e5-905c-7794efe7560e\") "
Sep 14 13:18:00.881180 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.881080    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-model-cache" (OuterVolumeSpecName: "model-cache") pod "9a5b773d-3a45-42e5-905c-7794efe7560e" (UID: "9a5b773d-3a45-42e5-905c-7794efe7560e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:00.881293 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.881257    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-home" (OuterVolumeSpecName: "home") pod "9a5b773d-3a45-42e5-905c-7794efe7560e" (UID: "9a5b773d-3a45-42e5-905c-7794efe7560e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:00.881352 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.881299    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9a5b773d-3a45-42e5-905c-7794efe7560e" (UID: "9a5b773d-3a45-42e5-905c-7794efe7560e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:00.881352 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.881324    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-model-cache\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:00.883245 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.883222    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-dshm" (OuterVolumeSpecName: "dshm") pod "9a5b773d-3a45-42e5-905c-7794efe7560e" (UID: "9a5b773d-3a45-42e5-905c-7794efe7560e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:00.883367 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.883270    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a5b773d-3a45-42e5-905c-7794efe7560e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9a5b773d-3a45-42e5-905c-7794efe7560e" (UID: "9a5b773d-3a45-42e5-905c-7794efe7560e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:18:00.883367 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.883275    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a5b773d-3a45-42e5-905c-7794efe7560e-kube-api-access-46l42" (OuterVolumeSpecName: "kube-api-access-46l42") pod "9a5b773d-3a45-42e5-905c-7794efe7560e" (UID: "9a5b773d-3a45-42e5-905c-7794efe7560e"). InnerVolumeSpecName "kube-api-access-46l42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:18:00.982597 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.982510    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a5b773d-3a45-42e5-905c-7794efe7560e-tls-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:00.982597 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.982590    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-home\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:00.982597 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.982601    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-tmp-dir\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:00.982597 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.982610    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46l42\" (UniqueName: \"kubernetes.io/projected/9a5b773d-3a45-42e5-905c-7794efe7560e-kube-api-access-46l42\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:00.982597 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.982618    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a5b773d-3a45-42e5-905c-7794efe7560e-dshm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:00.994355 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.994327    2581 generic.go:356] "Generic (PLEG): container finished" podID="9a5b773d-3a45-42e5-905c-7794efe7560e" containerID="6c8363157074aa2c785db3121f36fdcd9ce42b0ee6aee90c057eb3e83efcd5fa" exitCode=0
Sep 14 13:18:00.994486 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.994376    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"
Sep 14 13:18:00.994486 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.994392    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq" event={"ID":"9a5b773d-3a45-42e5-905c-7794efe7560e","Type":"ContainerDied","Data":"6c8363157074aa2c785db3121f36fdcd9ce42b0ee6aee90c057eb3e83efcd5fa"}
Sep 14 13:18:00.994486 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.994434    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq" event={"ID":"9a5b773d-3a45-42e5-905c-7794efe7560e","Type":"ContainerDied","Data":"c857a40d1289f71286bc356f3ff534c37f939e1670f219592040db992a432c7e"}
Sep 14 13:18:00.994486 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:00.994451    2581 scope.go:117] "RemoveContainer" containerID="6c8363157074aa2c785db3121f36fdcd9ce42b0ee6aee90c057eb3e83efcd5fa"
Sep 14 13:18:01.004754 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:01.004651    2581 scope.go:117] "RemoveContainer" containerID="6c8363157074aa2c785db3121f36fdcd9ce42b0ee6aee90c057eb3e83efcd5fa"
Sep 14 13:18:01.005032 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:01.004994    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c8363157074aa2c785db3121f36fdcd9ce42b0ee6aee90c057eb3e83efcd5fa\": container with ID starting with 6c8363157074aa2c785db3121f36fdcd9ce42b0ee6aee90c057eb3e83efcd5fa not found: ID does not exist" containerID="6c8363157074aa2c785db3121f36fdcd9ce42b0ee6aee90c057eb3e83efcd5fa"
Sep 14 13:18:01.005142 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:01.005033    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c8363157074aa2c785db3121f36fdcd9ce42b0ee6aee90c057eb3e83efcd5fa"} err="failed to get container status \"6c8363157074aa2c785db3121f36fdcd9ce42b0ee6aee90c057eb3e83efcd5fa\": rpc error: code = NotFound desc = could not find container \"6c8363157074aa2c785db3121f36fdcd9ce42b0ee6aee90c057eb3e83efcd5fa\": container with ID starting with 6c8363157074aa2c785db3121f36fdcd9ce42b0ee6aee90c057eb3e83efcd5fa not found: ID does not exist"
Sep 14 13:18:01.020729 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:01.020700    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"]
Sep 14 13:18:01.024801 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:01.024772    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-ace1c6/explicit-model-ace1c6-kserve-5f9f987b75-jbcsq"]
Sep 14 13:18:01.250719 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:01.248856    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a5b773d-3a45-42e5-905c-7794efe7560e" path="/var/lib/kubelet/pods/9a5b773d-3a45-42e5-905c-7794efe7560e/volumes"
Sep 14 13:18:01.919937 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:01.919889    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76" podUID="febdafa3-b3e3-41ec-820f-b1fcde95d32c" containerName="main" containerID="cri-o://7a96b8702551fd25e3ca11f177caa0a5c9381efe78e5082e382825d661674a3c" gracePeriod=45
Sep 14 13:18:02.170969 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.170907    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:18:02.295894 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.295857    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-tmp-dir\") pod \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") "
Sep 14 13:18:02.296101 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.295947    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-model-cache\") pod \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") "
Sep 14 13:18:02.296101 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.296002    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-dshm\") pod \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") "
Sep 14 13:18:02.296101 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.296027    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdj2h\" (UniqueName: \"kubernetes.io/projected/febdafa3-b3e3-41ec-820f-b1fcde95d32c-kube-api-access-mdj2h\") pod \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") "
Sep 14 13:18:02.296101 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.296054    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-home\") pod \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") "
Sep 14 13:18:02.296316 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.296115    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/febdafa3-b3e3-41ec-820f-b1fcde95d32c-tls-certs\") pod \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\" (UID: \"febdafa3-b3e3-41ec-820f-b1fcde95d32c\") "
Sep 14 13:18:02.296316 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.296167    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "febdafa3-b3e3-41ec-820f-b1fcde95d32c" (UID: "febdafa3-b3e3-41ec-820f-b1fcde95d32c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:02.296316 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.296258    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-model-cache" (OuterVolumeSpecName: "model-cache") pod "febdafa3-b3e3-41ec-820f-b1fcde95d32c" (UID: "febdafa3-b3e3-41ec-820f-b1fcde95d32c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:02.296460 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.296377    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-home" (OuterVolumeSpecName: "home") pod "febdafa3-b3e3-41ec-820f-b1fcde95d32c" (UID: "febdafa3-b3e3-41ec-820f-b1fcde95d32c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:02.296578 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.296557    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-home\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:02.296578 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.296578    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-tmp-dir\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:02.296746 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.296591    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-model-cache\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:02.298657 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.298625    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/febdafa3-b3e3-41ec-820f-b1fcde95d32c-kube-api-access-mdj2h" (OuterVolumeSpecName: "kube-api-access-mdj2h") pod "febdafa3-b3e3-41ec-820f-b1fcde95d32c" (UID: "febdafa3-b3e3-41ec-820f-b1fcde95d32c"). InnerVolumeSpecName "kube-api-access-mdj2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:18:02.298775 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.298668    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-dshm" (OuterVolumeSpecName: "dshm") pod "febdafa3-b3e3-41ec-820f-b1fcde95d32c" (UID: "febdafa3-b3e3-41ec-820f-b1fcde95d32c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:02.298775 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.298725    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/febdafa3-b3e3-41ec-820f-b1fcde95d32c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "febdafa3-b3e3-41ec-820f-b1fcde95d32c" (UID: "febdafa3-b3e3-41ec-820f-b1fcde95d32c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:18:02.397431 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.397390    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/febdafa3-b3e3-41ec-820f-b1fcde95d32c-dshm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:02.397431 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.397425    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdj2h\" (UniqueName: \"kubernetes.io/projected/febdafa3-b3e3-41ec-820f-b1fcde95d32c-kube-api-access-mdj2h\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:02.397431 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:02.397436    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/febdafa3-b3e3-41ec-820f-b1fcde95d32c-tls-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:03.007068 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:03.007034    2581 generic.go:356] "Generic (PLEG): container finished" podID="febdafa3-b3e3-41ec-820f-b1fcde95d32c" containerID="7a96b8702551fd25e3ca11f177caa0a5c9381efe78e5082e382825d661674a3c" exitCode=0
Sep 14 13:18:03.007822 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:03.007073    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"
Sep 14 13:18:03.007822 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:03.007121    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76" event={"ID":"febdafa3-b3e3-41ec-820f-b1fcde95d32c","Type":"ContainerDied","Data":"7a96b8702551fd25e3ca11f177caa0a5c9381efe78e5082e382825d661674a3c"}
Sep 14 13:18:03.007822 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:03.007173    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76" event={"ID":"febdafa3-b3e3-41ec-820f-b1fcde95d32c","Type":"ContainerDied","Data":"444c6e6e25f2a5c0b47689963a61494d212c31279acf59118b1faae2fd610152"}
Sep 14 13:18:03.007822 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:03.007194    2581 scope.go:117] "RemoveContainer" containerID="7a96b8702551fd25e3ca11f177caa0a5c9381efe78e5082e382825d661674a3c"
Sep 14 13:18:03.016742 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:03.016725    2581 scope.go:117] "RemoveContainer" containerID="7a96b8702551fd25e3ca11f177caa0a5c9381efe78e5082e382825d661674a3c"
Sep 14 13:18:03.017013 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:03.016997    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a96b8702551fd25e3ca11f177caa0a5c9381efe78e5082e382825d661674a3c\": container with ID starting with 7a96b8702551fd25e3ca11f177caa0a5c9381efe78e5082e382825d661674a3c not found: ID does not exist" containerID="7a96b8702551fd25e3ca11f177caa0a5c9381efe78e5082e382825d661674a3c"
Sep 14 13:18:03.017053 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:03.017021    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a96b8702551fd25e3ca11f177caa0a5c9381efe78e5082e382825d661674a3c"} err="failed to get container status \"7a96b8702551fd25e3ca11f177caa0a5c9381efe78e5082e382825d661674a3c\": rpc error: code = NotFound desc = could not find container \"7a96b8702551fd25e3ca11f177caa0a5c9381efe78e5082e382825d661674a3c\": container with ID starting with 7a96b8702551fd25e3ca11f177caa0a5c9381efe78e5082e382825d661674a3c not found: ID does not exist"
Sep 14 13:18:03.029784 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:03.029763    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"]
Sep 14 13:18:03.034037 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:03.034017    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-975227/test-model-975227-kserve-57cf5c6585-h5v76"]
Sep 14 13:18:03.247357 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:03.247312    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="febdafa3-b3e3-41ec-820f-b1fcde95d32c" path="/var/lib/kubelet/pods/febdafa3-b3e3-41ec-820f-b1fcde95d32c/volumes"
Sep 14 13:18:08.752042 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:08.752005    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:10.541842 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:10.541805    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:11.829258 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:11.829227    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:12.458255 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:12.458219    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:13.456881 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:13.456848    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:14.237849 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:14.237798    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:15.030822 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:15.030794    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:15.958327 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:15.958293    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:16.280640 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:16.280551    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:16.681145 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:16.681112    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:16.950196 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:16.950112    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:17.097820 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:17.097777    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:17.515682 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:17.515648    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:17.936847 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:17.936813    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:19.719115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:19.719087    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:20.567212 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:20.567176    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:20.657820 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:20.657789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:21.533284 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:21.533235    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:22.313046 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:22.313011    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:23.060625 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:23.060587    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:23.659343 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:23.659305    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:18:23.847436 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:23.847406    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:24.509136 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.509100    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"]
Sep 14 13:18:24.510153 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.510126    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69e8444f-e219-4301-8491-8ee0c412ed6f" containerName="payload-processing"
Sep 14 13:18:24.510153 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.510153    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="69e8444f-e219-4301-8491-8ee0c412ed6f" containerName="payload-processing"
Sep 14 13:18:24.510280 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.510165    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a5b773d-3a45-42e5-905c-7794efe7560e" containerName="main"
Sep 14 13:18:24.510280 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.510171    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a5b773d-3a45-42e5-905c-7794efe7560e" containerName="main"
Sep 14 13:18:24.510280 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.510191    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="febdafa3-b3e3-41ec-820f-b1fcde95d32c" containerName="main"
Sep 14 13:18:24.510280 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.510199    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="febdafa3-b3e3-41ec-820f-b1fcde95d32c" containerName="main"
Sep 14 13:18:24.510280 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.510279    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a5b773d-3a45-42e5-905c-7794efe7560e" containerName="main"
Sep 14 13:18:24.510521 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.510289    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="febdafa3-b3e3-41ec-820f-b1fcde95d32c" containerName="main"
Sep 14 13:18:24.510521 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.510302    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="69e8444f-e219-4301-8491-8ee0c412ed6f" containerName="payload-processing"
Sep 14 13:18:24.515297 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.515270    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:24.517766 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.517730    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-b8be88-serving-cert\""
Sep 14 13:18:24.521853 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.521822    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"]
Sep 14 13:18:24.550216 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.550182    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:24.618996 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.618950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtzl4\" (UniqueName: \"kubernetes.io/projected/a91be069-cf4b-4719-b643-435475df1c26-kube-api-access-xtzl4\") pod \"maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:24.619194 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.619080    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a91be069-cf4b-4719-b643-435475df1c26-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:24.619194 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.619152    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a91be069-cf4b-4719-b643-435475df1c26-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:24.619194 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.619172    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a91be069-cf4b-4719-b643-435475df1c26-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:24.633947 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.633894    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:24.720009 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.719974    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a91be069-cf4b-4719-b643-435475df1c26-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:24.720211 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.720044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtzl4\" (UniqueName: \"kubernetes.io/projected/a91be069-cf4b-4719-b643-435475df1c26-kube-api-access-xtzl4\") pod \"maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:24.720211 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.720105    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a91be069-cf4b-4719-b643-435475df1c26-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:24.720211 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.720138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a91be069-cf4b-4719-b643-435475df1c26-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:24.720892 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.720863    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a91be069-cf4b-4719-b643-435475df1c26-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:24.722621 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.722587    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a91be069-cf4b-4719-b643-435475df1c26-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:24.722759 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.722712    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a91be069-cf4b-4719-b643-435475df1c26-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:24.729642 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.729616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtzl4\" (UniqueName: \"kubernetes.io/projected/a91be069-cf4b-4719-b643-435475df1c26-kube-api-access-xtzl4\") pod \"maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:24.753446 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.753410    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:24.828997 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:24.828896    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:25.049312 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.049278    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:25.114333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.114297    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"]
Sep 14 13:18:25.114793 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:18:25.114768    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91be069_cf4b_4719_b643_435475df1c26.slice/crio-076f4dcb7b2923eb40a73ccab856fe521e7c987810d7e7f86a3e9d788e0008d5 WatchSource:0}: Error finding container 076f4dcb7b2923eb40a73ccab856fe521e7c987810d7e7f86a3e9d788e0008d5: Status 404 returned error can't find the container with id 076f4dcb7b2923eb40a73ccab856fe521e7c987810d7e7f86a3e9d788e0008d5
Sep 14 13:18:25.116439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.116422    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 13:18:25.131231 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.131198    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l" event={"ID":"a91be069-cf4b-4719-b643-435475df1c26","Type":"ContainerStarted","Data":"076f4dcb7b2923eb40a73ccab856fe521e7c987810d7e7f86a3e9d788e0008d5"}
Sep 14 13:18:25.167542 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.167510    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:25.353348 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.353307    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"]
Sep 14 13:18:25.358151 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.358123    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"
Sep 14 13:18:25.361797 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.361714    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-b8be88-dockercfg-f75jp\""
Sep 14 13:18:25.361975 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.361833    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-b8be88\""
Sep 14 13:18:25.382639 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.382602    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"]
Sep 14 13:18:25.428956 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.428911    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5hcd\" (UniqueName: \"kubernetes.io/projected/11b0249f-31cd-45d7-8540-cc3915a275a0-kube-api-access-b5hcd\") pod \"payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb\" (UID: \"11b0249f-31cd-45d7-8540-cc3915a275a0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"
Sep 14 13:18:25.429125 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.429072    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11b0249f-31cd-45d7-8540-cc3915a275a0-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb\" (UID: \"11b0249f-31cd-45d7-8540-cc3915a275a0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"
Sep 14 13:18:25.449837 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.449810    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:25.486901 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.486865    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"]
Sep 14 13:18:25.487160 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.487134    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q" podUID="20398542-b3f0-4df2-8485-d3fe43b8aaba" containerName="payload-processing" containerID="cri-o://322845275d7c60e3c1c9c4ace61fb1ea43cf0331b34664a27ae50efd71e8edfa" gracePeriod=30
Sep 14 13:18:25.503084 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.503011    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"]
Sep 14 13:18:25.504407 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.503229    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54" podUID="8d56be6b-d25a-4727-b0e6-5b6baef100da" containerName="payload-pre-processing" containerID="cri-o://e190268ccbeae8e704088266faf63af1956da58730c29be0d35dd89f33e8a53f" gracePeriod=30
Sep 14 13:18:25.530078 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.530036    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5hcd\" (UniqueName: \"kubernetes.io/projected/11b0249f-31cd-45d7-8540-cc3915a275a0-kube-api-access-b5hcd\") pod \"payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb\" (UID: \"11b0249f-31cd-45d7-8540-cc3915a275a0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"
Sep 14 13:18:25.530550 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.530102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11b0249f-31cd-45d7-8540-cc3915a275a0-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb\" (UID: \"11b0249f-31cd-45d7-8540-cc3915a275a0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"
Sep 14 13:18:25.530872 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.530851    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11b0249f-31cd-45d7-8540-cc3915a275a0-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb\" (UID: \"11b0249f-31cd-45d7-8540-cc3915a275a0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"
Sep 14 13:18:25.536594 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.536551    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:25.542059 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.542029    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5hcd\" (UniqueName: \"kubernetes.io/projected/11b0249f-31cd-45d7-8540-cc3915a275a0-kube-api-access-b5hcd\") pod \"payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb\" (UID: \"11b0249f-31cd-45d7-8540-cc3915a275a0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"
Sep 14 13:18:25.644390 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.643829    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:25.669331 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.669296    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"
Sep 14 13:18:25.755142 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.755078    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:25.813242 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.813213    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"
Sep 14 13:18:25.815679 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.815653    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"
Sep 14 13:18:25.874321 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.874231    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:25.933853 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.933794    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d56be6b-d25a-4727-b0e6-5b6baef100da-plugins-config-volume\") pod \"8d56be6b-d25a-4727-b0e6-5b6baef100da\" (UID: \"8d56be6b-d25a-4727-b0e6-5b6baef100da\") "
Sep 14 13:18:25.934081 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.933876    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgxnz\" (UniqueName: \"kubernetes.io/projected/8d56be6b-d25a-4727-b0e6-5b6baef100da-kube-api-access-jgxnz\") pod \"8d56be6b-d25a-4727-b0e6-5b6baef100da\" (UID: \"8d56be6b-d25a-4727-b0e6-5b6baef100da\") "
Sep 14 13:18:25.934081 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.933912    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20398542-b3f0-4df2-8485-d3fe43b8aaba-plugins-config-volume\") pod \"20398542-b3f0-4df2-8485-d3fe43b8aaba\" (UID: \"20398542-b3f0-4df2-8485-d3fe43b8aaba\") "
Sep 14 13:18:25.934081 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.933980    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jtmk\" (UniqueName: \"kubernetes.io/projected/20398542-b3f0-4df2-8485-d3fe43b8aaba-kube-api-access-4jtmk\") pod \"20398542-b3f0-4df2-8485-d3fe43b8aaba\" (UID: \"20398542-b3f0-4df2-8485-d3fe43b8aaba\") "
Sep 14 13:18:25.934255 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.934167    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d56be6b-d25a-4727-b0e6-5b6baef100da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8d56be6b-d25a-4727-b0e6-5b6baef100da" (UID: "8d56be6b-d25a-4727-b0e6-5b6baef100da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:18:25.934387 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.934355    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20398542-b3f0-4df2-8485-d3fe43b8aaba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20398542-b3f0-4df2-8485-d3fe43b8aaba" (UID: "20398542-b3f0-4df2-8485-d3fe43b8aaba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:18:25.936214 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.936191    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d56be6b-d25a-4727-b0e6-5b6baef100da-kube-api-access-jgxnz" (OuterVolumeSpecName: "kube-api-access-jgxnz") pod "8d56be6b-d25a-4727-b0e6-5b6baef100da" (UID: "8d56be6b-d25a-4727-b0e6-5b6baef100da"). InnerVolumeSpecName "kube-api-access-jgxnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:18:25.936329 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.936309    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20398542-b3f0-4df2-8485-d3fe43b8aaba-kube-api-access-4jtmk" (OuterVolumeSpecName: "kube-api-access-4jtmk") pod "20398542-b3f0-4df2-8485-d3fe43b8aaba" (UID: "20398542-b3f0-4df2-8485-d3fe43b8aaba"). InnerVolumeSpecName "kube-api-access-4jtmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:18:25.965348 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:25.965322    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"]
Sep 14 13:18:25.966934 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:18:25.966900    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11b0249f_31cd_45d7_8540_cc3915a275a0.slice/crio-3d4b9b096b2711ebde2a95a4c08324c6196757a739096f2fc7b7176903a81c9a WatchSource:0}: Error finding container 3d4b9b096b2711ebde2a95a4c08324c6196757a739096f2fc7b7176903a81c9a: Status 404 returned error can't find the container with id 3d4b9b096b2711ebde2a95a4c08324c6196757a739096f2fc7b7176903a81c9a
Sep 14 13:18:26.034916 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.034894    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgxnz\" (UniqueName: \"kubernetes.io/projected/8d56be6b-d25a-4727-b0e6-5b6baef100da-kube-api-access-jgxnz\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:26.035007 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.034918    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20398542-b3f0-4df2-8485-d3fe43b8aaba-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:26.035007 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.034927    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jtmk\" (UniqueName: \"kubernetes.io/projected/20398542-b3f0-4df2-8485-d3fe43b8aaba-kube-api-access-4jtmk\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:26.035007 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.034935    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d56be6b-d25a-4727-b0e6-5b6baef100da-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:26.124626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.124516    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:26.138121 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.138085    2581 generic.go:356] "Generic (PLEG): container finished" podID="20398542-b3f0-4df2-8485-d3fe43b8aaba" containerID="322845275d7c60e3c1c9c4ace61fb1ea43cf0331b34664a27ae50efd71e8edfa" exitCode=0
Sep 14 13:18:26.138297 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.138129    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"
Sep 14 13:18:26.138297 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.138175    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q" event={"ID":"20398542-b3f0-4df2-8485-d3fe43b8aaba","Type":"ContainerDied","Data":"322845275d7c60e3c1c9c4ace61fb1ea43cf0331b34664a27ae50efd71e8edfa"}
Sep 14 13:18:26.138297 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.138232    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q" event={"ID":"20398542-b3f0-4df2-8485-d3fe43b8aaba","Type":"ContainerDied","Data":"b3eb96166abec0e5dd76e37fcec2d8973ed26c124e30238075e38cdcaea704f1"}
Sep 14 13:18:26.138297 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.138262    2581 scope.go:117] "RemoveContainer" containerID="322845275d7c60e3c1c9c4ace61fb1ea43cf0331b34664a27ae50efd71e8edfa"
Sep 14 13:18:26.140116 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.140089    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb" event={"ID":"11b0249f-31cd-45d7-8540-cc3915a275a0","Type":"ContainerStarted","Data":"7d908c65dc032d832e5692c0259107aea0be6d076f598b1aa499df76bef73aa3"}
Sep 14 13:18:26.140251 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.140152    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"
Sep 14 13:18:26.140251 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.140176    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb" event={"ID":"11b0249f-31cd-45d7-8540-cc3915a275a0","Type":"ContainerStarted","Data":"3d4b9b096b2711ebde2a95a4c08324c6196757a739096f2fc7b7176903a81c9a"}
Sep 14 13:18:26.151458 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.151397    2581 generic.go:356] "Generic (PLEG): container finished" podID="8d56be6b-d25a-4727-b0e6-5b6baef100da" containerID="e190268ccbeae8e704088266faf63af1956da58730c29be0d35dd89f33e8a53f" exitCode=0
Sep 14 13:18:26.151726 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.151478    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"
Sep 14 13:18:26.151726 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.151559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54" event={"ID":"8d56be6b-d25a-4727-b0e6-5b6baef100da","Type":"ContainerDied","Data":"e190268ccbeae8e704088266faf63af1956da58730c29be0d35dd89f33e8a53f"}
Sep 14 13:18:26.151726 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.151637    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54" event={"ID":"8d56be6b-d25a-4727-b0e6-5b6baef100da","Type":"ContainerDied","Data":"f3352374ee0ea5ce2d1b2c64019967193b8c81fd4ac7514dd72128335dfd2e98"}
Sep 14 13:18:26.154035 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.154006    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l" event={"ID":"a91be069-cf4b-4719-b643-435475df1c26","Type":"ContainerStarted","Data":"36a01c20dbdc02155c27b9eda9aaaf176452935375bf3a476ce2a727653542fd"}
Sep 14 13:18:26.154515 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.154498    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:26.157131 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.157116    2581 scope.go:117] "RemoveContainer" containerID="322845275d7c60e3c1c9c4ace61fb1ea43cf0331b34664a27ae50efd71e8edfa"
Sep 14 13:18:26.157409 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:26.157391    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"322845275d7c60e3c1c9c4ace61fb1ea43cf0331b34664a27ae50efd71e8edfa\": container with ID starting with 322845275d7c60e3c1c9c4ace61fb1ea43cf0331b34664a27ae50efd71e8edfa not found: ID does not exist" containerID="322845275d7c60e3c1c9c4ace61fb1ea43cf0331b34664a27ae50efd71e8edfa"
Sep 14 13:18:26.157461 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.157417    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"322845275d7c60e3c1c9c4ace61fb1ea43cf0331b34664a27ae50efd71e8edfa"} err="failed to get container status \"322845275d7c60e3c1c9c4ace61fb1ea43cf0331b34664a27ae50efd71e8edfa\": rpc error: code = NotFound desc = could not find container \"322845275d7c60e3c1c9c4ace61fb1ea43cf0331b34664a27ae50efd71e8edfa\": container with ID starting with 322845275d7c60e3c1c9c4ace61fb1ea43cf0331b34664a27ae50efd71e8edfa not found: ID does not exist"
Sep 14 13:18:26.157461 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.157433    2581 scope.go:117] "RemoveContainer" containerID="e190268ccbeae8e704088266faf63af1956da58730c29be0d35dd89f33e8a53f"
Sep 14 13:18:26.167183 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.167156    2581 scope.go:117] "RemoveContainer" containerID="e190268ccbeae8e704088266faf63af1956da58730c29be0d35dd89f33e8a53f"
Sep 14 13:18:26.167522 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:26.167502    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e190268ccbeae8e704088266faf63af1956da58730c29be0d35dd89f33e8a53f\": container with ID starting with e190268ccbeae8e704088266faf63af1956da58730c29be0d35dd89f33e8a53f not found: ID does not exist" containerID="e190268ccbeae8e704088266faf63af1956da58730c29be0d35dd89f33e8a53f"
Sep 14 13:18:26.167614 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.167546    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e190268ccbeae8e704088266faf63af1956da58730c29be0d35dd89f33e8a53f"} err="failed to get container status \"e190268ccbeae8e704088266faf63af1956da58730c29be0d35dd89f33e8a53f\": rpc error: code = NotFound desc = could not find container \"e190268ccbeae8e704088266faf63af1956da58730c29be0d35dd89f33e8a53f\": container with ID starting with e190268ccbeae8e704088266faf63af1956da58730c29be0d35dd89f33e8a53f not found: ID does not exist"
Sep 14 13:18:26.173015 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.172967    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb" podStartSLOduration=1.17295203 podStartE2EDuration="1.17295203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:18:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:18:26.171796285 +0000 UTC m=+1199.463130099" watchObservedRunningTime="2026-09-14 13:18:26.17295203 +0000 UTC m=+1199.464285848"
Sep 14 13:18:26.189982 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.189949    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"]
Sep 14 13:18:26.198309 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.198274    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-975227-79cf86b4b9-2rk8q"]
Sep 14 13:18:26.224978 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.224923    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l" podStartSLOduration=1.897263355 podStartE2EDuration="2.224905214s" totalImagesPullingTime="327.641859ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:18:26.223457996 +0000 UTC m=+1199.514791814" watchObservedRunningTime="2026-09-14 13:18:26.224905214 +0000 UTC m=+1199.516239031"
Sep 14 13:18:26.249362 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.249323    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"]
Sep 14 13:18:26.259900 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.259871    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-975227-544d759b7d-c8k54"]
Sep 14 13:18:26.554933 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:26.554849    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:27.217954 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:27.217926    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:18:27.219234 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:27.219211    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:18:27.245935 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:27.245901    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20398542-b3f0-4df2-8485-d3fe43b8aaba" path="/var/lib/kubelet/pods/20398542-b3f0-4df2-8485-d3fe43b8aaba/volumes"
Sep 14 13:18:27.246286 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:27.246272    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d56be6b-d25a-4727-b0e6-5b6baef100da" path="/var/lib/kubelet/pods/8d56be6b-d25a-4727-b0e6-5b6baef100da/volumes"
Sep 14 13:18:27.520306 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:27.520219    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:28.349271 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:28.349239    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:29.181898 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:29.181865    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:30.082751 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:30.082719    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:30.828832 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:30.828801    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:32.624544 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:32.624491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:32.869510 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:32.869475    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:33.172781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:33.172751    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:18:33.296502 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:33.296469    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:33.383163 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:33.383132    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:33.627442 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:33.627410    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:33.730626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:33.730593    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:34.089008 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:34.088921    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:34.239562 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:34.239491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:35.009597 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:35.009567    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:35.116576 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:35.116544    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:35.227851 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:35.227805    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:35.315856 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:35.315772    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:35.663171 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:35.663135    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:36.107718 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:36.107668    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:37.126348 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:37.126301    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:37.164315 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:37.164286    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"
Sep 14 13:18:38.331990 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:38.331954    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:39.265868 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:39.265832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:40.041745 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:40.041714    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:40.349150 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:40.349071    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:40.780004 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:40.779969    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:41.114435 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.114398    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7419-e267b2"]
Sep 14 13:18:41.114875 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.114812    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20398542-b3f0-4df2-8485-d3fe43b8aaba" containerName="payload-processing"
Sep 14 13:18:41.114875 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.114826    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="20398542-b3f0-4df2-8485-d3fe43b8aaba" containerName="payload-processing"
Sep 14 13:18:41.114875 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.114840    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d56be6b-d25a-4727-b0e6-5b6baef100da" containerName="payload-pre-processing"
Sep 14 13:18:41.114875 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.114845    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d56be6b-d25a-4727-b0e6-5b6baef100da" containerName="payload-pre-processing"
Sep 14 13:18:41.115018 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.114917    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="20398542-b3f0-4df2-8485-d3fe43b8aaba" containerName="payload-processing"
Sep 14 13:18:41.115018 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.114928    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d56be6b-d25a-4727-b0e6-5b6baef100da" containerName="payload-pre-processing"
Sep 14 13:18:41.119927 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.119909    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-e267b2"
Sep 14 13:18:41.122617 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.122582    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vtxmg\""
Sep 14 13:18:41.123651 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.123629    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7419-e267b2"]
Sep 14 13:18:41.195318 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.195280    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tqqz\" (UniqueName: \"kubernetes.io/projected/b30081d9-0975-40f7-a79c-d77e04096d77-kube-api-access-2tqqz\") pod \"test-curl-7419-e267b2\" (UID: \"b30081d9-0975-40f7-a79c-d77e04096d77\") " pod="opendatahub/test-curl-7419-e267b2"
Sep 14 13:18:41.296262 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.296221    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tqqz\" (UniqueName: \"kubernetes.io/projected/b30081d9-0975-40f7-a79c-d77e04096d77-kube-api-access-2tqqz\") pod \"test-curl-7419-e267b2\" (UID: \"b30081d9-0975-40f7-a79c-d77e04096d77\") " pod="opendatahub/test-curl-7419-e267b2"
Sep 14 13:18:41.306433 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.306403    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tqqz\" (UniqueName: \"kubernetes.io/projected/b30081d9-0975-40f7-a79c-d77e04096d77-kube-api-access-2tqqz\") pod \"test-curl-7419-e267b2\" (UID: \"b30081d9-0975-40f7-a79c-d77e04096d77\") " pod="opendatahub/test-curl-7419-e267b2"
Sep 14 13:18:41.437417 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.437331    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-e267b2"
Sep 14 13:18:41.583900 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.583870    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:41.713440 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:41.713309    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7419-e267b2"]
Sep 14 13:18:41.717147 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:18:41.717116    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb30081d9_0975_40f7_a79c_d77e04096d77.slice/crio-0252653d98fa19f37a3734f2aa5e9d4350ff3c0edb74cd8f1334f4ce888377c5 WatchSource:0}: Error finding container 0252653d98fa19f37a3734f2aa5e9d4350ff3c0edb74cd8f1334f4ce888377c5: Status 404 returned error can't find the container with id 0252653d98fa19f37a3734f2aa5e9d4350ff3c0edb74cd8f1334f4ce888377c5
Sep 14 13:18:42.244898 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:42.244855    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7419-e267b2" event={"ID":"b30081d9-0975-40f7-a79c-d77e04096d77","Type":"ContainerStarted","Data":"be9b4770a3f9d075e7367cb7ddb9d369c6636c01587c987e992cecf06cc3908d"}
Sep 14 13:18:42.244898 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:42.244901    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7419-e267b2" event={"ID":"b30081d9-0975-40f7-a79c-d77e04096d77","Type":"ContainerStarted","Data":"0252653d98fa19f37a3734f2aa5e9d4350ff3c0edb74cd8f1334f4ce888377c5"}
Sep 14 13:18:42.257500 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:42.257472    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:42.261725 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:42.261681    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7419-e267b2" podStartSLOduration=1.261667635 podStartE2EDuration="1.261667635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:18:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:18:42.259667726 +0000 UTC m=+1215.551001544" watchObservedRunningTime="2026-09-14 13:18:42.261667635 +0000 UTC m=+1215.553001454"
Sep 14 13:18:42.935080 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:42.935049    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7419-e267b2"]
Sep 14 13:18:42.937475 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:42.937398    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7419-e267b2"]
Sep 14 13:18:43.053584 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:43.053543    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7419-fac83c"]
Sep 14 13:18:43.054030 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:43.054012    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b30081d9-0975-40f7-a79c-d77e04096d77" containerName="test-curl-7419-e267b2"
Sep 14 13:18:43.054123 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:43.054034    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b30081d9-0975-40f7-a79c-d77e04096d77" containerName="test-curl-7419-e267b2"
Sep 14 13:18:43.054176 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:43.054155    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b30081d9-0975-40f7-a79c-d77e04096d77" containerName="test-curl-7419-e267b2"
Sep 14 13:18:43.057339 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:43.057319    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-fac83c"
Sep 14 13:18:43.064883 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:43.064850    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7419-fac83c"]
Sep 14 13:18:43.216579 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:43.216466    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6pjw\" (UniqueName: \"kubernetes.io/projected/762e051d-2522-4ee3-b91f-61292546bc75-kube-api-access-k6pjw\") pod \"test-curl-7419-fac83c\" (UID: \"762e051d-2522-4ee3-b91f-61292546bc75\") " pod="opendatahub/test-curl-7419-fac83c"
Sep 14 13:18:43.250423 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:43.250392    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7419-e267b2" container="test-curl-7419-e267b2"
Sep 14 13:18:43.317291 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:43.317251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6pjw\" (UniqueName: \"kubernetes.io/projected/762e051d-2522-4ee3-b91f-61292546bc75-kube-api-access-k6pjw\") pod \"test-curl-7419-fac83c\" (UID: \"762e051d-2522-4ee3-b91f-61292546bc75\") " pod="opendatahub/test-curl-7419-fac83c"
Sep 14 13:18:43.328470 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:43.328432    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6pjw\" (UniqueName: \"kubernetes.io/projected/762e051d-2522-4ee3-b91f-61292546bc75-kube-api-access-k6pjw\") pod \"test-curl-7419-fac83c\" (UID: \"762e051d-2522-4ee3-b91f-61292546bc75\") " pod="opendatahub/test-curl-7419-fac83c"
Sep 14 13:18:43.346476 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:43.346441    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:43.367301 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:43.367268    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-fac83c"
Sep 14 13:18:43.640806 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:43.640779    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7419-fac83c"]
Sep 14 13:18:43.642856 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:18:43.642821    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod762e051d_2522_4ee3_b91f_61292546bc75.slice/crio-e7f8b06ca228fe1f608fb94ab35246c24c27806babb69020bab605ce7e771724 WatchSource:0}: Error finding container e7f8b06ca228fe1f608fb94ab35246c24c27806babb69020bab605ce7e771724: Status 404 returned error can't find the container with id e7f8b06ca228fe1f608fb94ab35246c24c27806babb69020bab605ce7e771724
Sep 14 13:18:44.148210 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.148176    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"]
Sep 14 13:18:44.148418 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.148394    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v" podUID="ba5f701e-637f-44b4-8b4a-090d42179f7e" containerName="maas-api" containerID="cri-o://a3e124e7a3bdf0b35e42172bd246ac56cf6a4ade59bf6b48a3230c88f6a401ec" gracePeriod=30
Sep 14 13:18:44.191683 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.191649    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:44.226371 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.226336    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"]
Sep 14 13:18:44.226628 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.226594    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9" podUID="23719951-7853-459f-afed-b3ba7c2b8660" containerName="payload-processing" containerID="cri-o://c1c755b4f86a7b199361b610eefcea49a2650e8cc95a82be257ff0b8423dffd0" gracePeriod=30
Sep 14 13:18:44.231605 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.231581    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"]
Sep 14 13:18:44.232145 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.232052    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q" podUID="af5c1011-5523-466c-b844-787991bce9db" containerName="payload-pre-processing" containerID="cri-o://0e484f3566f0fbc8c03b28c1c0ffb9e7694b3f547788724f93c565939a17e846" gracePeriod=30
Sep 14 13:18:44.261160 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.261119    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7419-fac83c" event={"ID":"762e051d-2522-4ee3-b91f-61292546bc75","Type":"ContainerStarted","Data":"30bcb6701ea7de38d3a520097ed7dd262c37f1dc63bfa4d22237b27715278190"}
Sep 14 13:18:44.261570 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.261175    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7419-fac83c" event={"ID":"762e051d-2522-4ee3-b91f-61292546bc75","Type":"ContainerStarted","Data":"e7f8b06ca228fe1f608fb94ab35246c24c27806babb69020bab605ce7e771724"}
Sep 14 13:18:44.261570 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.261250    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7419-e267b2" podUID="b30081d9-0975-40f7-a79c-d77e04096d77" containerName="test-curl-7419-e267b2" containerID="cri-o://be9b4770a3f9d075e7367cb7ddb9d369c6636c01587c987e992cecf06cc3908d" gracePeriod=2
Sep 14 13:18:44.300948 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.300918    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:44.310757 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.310699    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7419-fac83c" podStartSLOduration=1.3106822 podStartE2EDuration="1.3106822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:18:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:18:44.309585094 +0000 UTC m=+1217.600918911" watchObservedRunningTime="2026-09-14 13:18:44.3106822 +0000 UTC m=+1217.602016017"
Sep 14 13:18:44.424209 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.424034    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:44.488614 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.488582    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"
Sep 14 13:18:44.533813 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.533783    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:18:44.535234 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.535209    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:44.537202 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.537183    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"
Sep 14 13:18:44.628409 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.628379    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns475\" (UniqueName: \"kubernetes.io/projected/af5c1011-5523-466c-b844-787991bce9db-kube-api-access-ns475\") pod \"af5c1011-5523-466c-b844-787991bce9db\" (UID: \"af5c1011-5523-466c-b844-787991bce9db\") "
Sep 14 13:18:44.628622 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.628476    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba5f701e-637f-44b4-8b4a-090d42179f7e-metrics-certs\") pod \"ba5f701e-637f-44b4-8b4a-090d42179f7e\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") "
Sep 14 13:18:44.628622 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.628506    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23719951-7853-459f-afed-b3ba7c2b8660-plugins-config-volume\") pod \"23719951-7853-459f-afed-b3ba7c2b8660\" (UID: \"23719951-7853-459f-afed-b3ba7c2b8660\") "
Sep 14 13:18:44.628622 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.628580    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af5c1011-5523-466c-b844-787991bce9db-plugins-config-volume\") pod \"af5c1011-5523-466c-b844-787991bce9db\" (UID: \"af5c1011-5523-466c-b844-787991bce9db\") "
Sep 14 13:18:44.628808 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.628695    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqr6w\" (UniqueName: \"kubernetes.io/projected/23719951-7853-459f-afed-b3ba7c2b8660-kube-api-access-dqr6w\") pod \"23719951-7853-459f-afed-b3ba7c2b8660\" (UID: \"23719951-7853-459f-afed-b3ba7c2b8660\") "
Sep 14 13:18:44.628808 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.628741    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba5f701e-637f-44b4-8b4a-090d42179f7e-maas-api-tls\") pod \"ba5f701e-637f-44b4-8b4a-090d42179f7e\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") "
Sep 14 13:18:44.628910 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.628807    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba5f701e-637f-44b4-8b4a-090d42179f7e-openshift-service-ca\") pod \"ba5f701e-637f-44b4-8b4a-090d42179f7e\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") "
Sep 14 13:18:44.628910 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.628844    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qk4d\" (UniqueName: \"kubernetes.io/projected/ba5f701e-637f-44b4-8b4a-090d42179f7e-kube-api-access-8qk4d\") pod \"ba5f701e-637f-44b4-8b4a-090d42179f7e\" (UID: \"ba5f701e-637f-44b4-8b4a-090d42179f7e\") "
Sep 14 13:18:44.629010 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.628946    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af5c1011-5523-466c-b844-787991bce9db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "af5c1011-5523-466c-b844-787991bce9db" (UID: "af5c1011-5523-466c-b844-787991bce9db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:18:44.629010 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.628961    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23719951-7853-459f-afed-b3ba7c2b8660-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "23719951-7853-459f-afed-b3ba7c2b8660" (UID: "23719951-7853-459f-afed-b3ba7c2b8660"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:18:44.629461 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.629405    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23719951-7853-459f-afed-b3ba7c2b8660-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:44.629461 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.629443    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af5c1011-5523-466c-b844-787991bce9db-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:44.629461 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.629390    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba5f701e-637f-44b4-8b4a-090d42179f7e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ba5f701e-637f-44b4-8b4a-090d42179f7e" (UID: "ba5f701e-637f-44b4-8b4a-090d42179f7e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:18:44.630766 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.630738    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af5c1011-5523-466c-b844-787991bce9db-kube-api-access-ns475" (OuterVolumeSpecName: "kube-api-access-ns475") pod "af5c1011-5523-466c-b844-787991bce9db" (UID: "af5c1011-5523-466c-b844-787991bce9db"). InnerVolumeSpecName "kube-api-access-ns475". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:18:44.631188 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.631162    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba5f701e-637f-44b4-8b4a-090d42179f7e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ba5f701e-637f-44b4-8b4a-090d42179f7e" (UID: "ba5f701e-637f-44b4-8b4a-090d42179f7e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:18:44.631188 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.631171    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba5f701e-637f-44b4-8b4a-090d42179f7e-kube-api-access-8qk4d" (OuterVolumeSpecName: "kube-api-access-8qk4d") pod "ba5f701e-637f-44b4-8b4a-090d42179f7e" (UID: "ba5f701e-637f-44b4-8b4a-090d42179f7e"). InnerVolumeSpecName "kube-api-access-8qk4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:18:44.631448 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.631431    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba5f701e-637f-44b4-8b4a-090d42179f7e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ba5f701e-637f-44b4-8b4a-090d42179f7e" (UID: "ba5f701e-637f-44b4-8b4a-090d42179f7e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:18:44.631558 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.631433    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23719951-7853-459f-afed-b3ba7c2b8660-kube-api-access-dqr6w" (OuterVolumeSpecName: "kube-api-access-dqr6w") pod "23719951-7853-459f-afed-b3ba7c2b8660" (UID: "23719951-7853-459f-afed-b3ba7c2b8660"). InnerVolumeSpecName "kube-api-access-dqr6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:18:44.730513 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.730412    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns475\" (UniqueName: \"kubernetes.io/projected/af5c1011-5523-466c-b844-787991bce9db-kube-api-access-ns475\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:44.730513 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.730444    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba5f701e-637f-44b4-8b4a-090d42179f7e-metrics-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:44.730513 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.730459    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqr6w\" (UniqueName: \"kubernetes.io/projected/23719951-7853-459f-afed-b3ba7c2b8660-kube-api-access-dqr6w\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:44.730513 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.730472    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba5f701e-637f-44b4-8b4a-090d42179f7e-maas-api-tls\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:44.730513 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.730483    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba5f701e-637f-44b4-8b4a-090d42179f7e-openshift-service-ca\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:44.730513 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.730494    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qk4d\" (UniqueName: \"kubernetes.io/projected/ba5f701e-637f-44b4-8b4a-090d42179f7e-kube-api-access-8qk4d\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:44.934069 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:44.934041    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:45.104090 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.104055    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7419-fac83c"]
Sep 14 13:18:45.113153 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.113124    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7419-fac83c"]
Sep 14 13:18:45.225910 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.225874    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7419-9e826f"]
Sep 14 13:18:45.226321 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.226307    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23719951-7853-459f-afed-b3ba7c2b8660" containerName="payload-processing"
Sep 14 13:18:45.226385 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.226323    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="23719951-7853-459f-afed-b3ba7c2b8660" containerName="payload-processing"
Sep 14 13:18:45.226385 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.226336    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba5f701e-637f-44b4-8b4a-090d42179f7e" containerName="maas-api"
Sep 14 13:18:45.226385 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.226342    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba5f701e-637f-44b4-8b4a-090d42179f7e" containerName="maas-api"
Sep 14 13:18:45.226385 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.226354    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="762e051d-2522-4ee3-b91f-61292546bc75" containerName="test-curl-7419-fac83c"
Sep 14 13:18:45.226385 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.226360    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="762e051d-2522-4ee3-b91f-61292546bc75" containerName="test-curl-7419-fac83c"
Sep 14 13:18:45.226385 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.226371    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af5c1011-5523-466c-b844-787991bce9db" containerName="payload-pre-processing"
Sep 14 13:18:45.226385 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.226376    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="af5c1011-5523-466c-b844-787991bce9db" containerName="payload-pre-processing"
Sep 14 13:18:45.226640 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.226433    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="23719951-7853-459f-afed-b3ba7c2b8660" containerName="payload-processing"
Sep 14 13:18:45.226640 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.226442    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="762e051d-2522-4ee3-b91f-61292546bc75" containerName="test-curl-7419-fac83c"
Sep 14 13:18:45.226640 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.226448    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="af5c1011-5523-466c-b844-787991bce9db" containerName="payload-pre-processing"
Sep 14 13:18:45.226640 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.226455    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba5f701e-637f-44b4-8b4a-090d42179f7e" containerName="maas-api"
Sep 14 13:18:45.230809 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.230788    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:45.248851 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.248823    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7419-9e826f"]
Sep 14 13:18:45.267799 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.267773    2581 generic.go:356] "Generic (PLEG): container finished" podID="af5c1011-5523-466c-b844-787991bce9db" containerID="0e484f3566f0fbc8c03b28c1c0ffb9e7694b3f547788724f93c565939a17e846" exitCode=0
Sep 14 13:18:45.268130 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.267815    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"
Sep 14 13:18:45.268130 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.267855    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q" event={"ID":"af5c1011-5523-466c-b844-787991bce9db","Type":"ContainerDied","Data":"0e484f3566f0fbc8c03b28c1c0ffb9e7694b3f547788724f93c565939a17e846"}
Sep 14 13:18:45.268130 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.267902    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q" event={"ID":"af5c1011-5523-466c-b844-787991bce9db","Type":"ContainerDied","Data":"cef3603e0b1c9e0d734596b3e3bad628cf7425b957d30349677e678bcc3a663e"}
Sep 14 13:18:45.268130 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.267931    2581 scope.go:117] "RemoveContainer" containerID="0e484f3566f0fbc8c03b28c1c0ffb9e7694b3f547788724f93c565939a17e846"
Sep 14 13:18:45.269374 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.269349    2581 generic.go:356] "Generic (PLEG): container finished" podID="ba5f701e-637f-44b4-8b4a-090d42179f7e" containerID="a3e124e7a3bdf0b35e42172bd246ac56cf6a4ade59bf6b48a3230c88f6a401ec" exitCode=0
Sep 14 13:18:45.269456 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.269388    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"
Sep 14 13:18:45.269456 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.269433    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v" event={"ID":"ba5f701e-637f-44b4-8b4a-090d42179f7e","Type":"ContainerDied","Data":"a3e124e7a3bdf0b35e42172bd246ac56cf6a4ade59bf6b48a3230c88f6a401ec"}
Sep 14 13:18:45.269598 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.269484    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v" event={"ID":"ba5f701e-637f-44b4-8b4a-090d42179f7e","Type":"ContainerDied","Data":"5678fad21acf10a61714c6294e983ece8530290db7204e2ba85e8ddf0d022215"}
Sep 14 13:18:45.271498 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.271478    2581 generic.go:356] "Generic (PLEG): container finished" podID="23719951-7853-459f-afed-b3ba7c2b8660" containerID="c1c755b4f86a7b199361b610eefcea49a2650e8cc95a82be257ff0b8423dffd0" exitCode=0
Sep 14 13:18:45.271771 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.271502    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9" event={"ID":"23719951-7853-459f-afed-b3ba7c2b8660","Type":"ContainerDied","Data":"c1c755b4f86a7b199361b610eefcea49a2650e8cc95a82be257ff0b8423dffd0"}
Sep 14 13:18:45.271861 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.271818    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9" event={"ID":"23719951-7853-459f-afed-b3ba7c2b8660","Type":"ContainerDied","Data":"0c32933ae0a2d923dfa94ab5b17b7a1bd34f3d75b9d46b7d3e97bb3ebd18c793"}
Sep 14 13:18:45.271861 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.271558    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"
Sep 14 13:18:45.272161 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:45.272143    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7419-fac83c" container="test-curl-7419-fac83c"
Sep 14 13:18:45.280307 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.280288    2581 scope.go:117] "RemoveContainer" containerID="0e484f3566f0fbc8c03b28c1c0ffb9e7694b3f547788724f93c565939a17e846"
Sep 14 13:18:45.280912 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:45.280886    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e484f3566f0fbc8c03b28c1c0ffb9e7694b3f547788724f93c565939a17e846\": container with ID starting with 0e484f3566f0fbc8c03b28c1c0ffb9e7694b3f547788724f93c565939a17e846 not found: ID does not exist" containerID="0e484f3566f0fbc8c03b28c1c0ffb9e7694b3f547788724f93c565939a17e846"
Sep 14 13:18:45.280991 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.280919    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e484f3566f0fbc8c03b28c1c0ffb9e7694b3f547788724f93c565939a17e846"} err="failed to get container status \"0e484f3566f0fbc8c03b28c1c0ffb9e7694b3f547788724f93c565939a17e846\": rpc error: code = NotFound desc = could not find container \"0e484f3566f0fbc8c03b28c1c0ffb9e7694b3f547788724f93c565939a17e846\": container with ID starting with 0e484f3566f0fbc8c03b28c1c0ffb9e7694b3f547788724f93c565939a17e846 not found: ID does not exist"
Sep 14 13:18:45.280991 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.280936    2581 scope.go:117] "RemoveContainer" containerID="a3e124e7a3bdf0b35e42172bd246ac56cf6a4ade59bf6b48a3230c88f6a401ec"
Sep 14 13:18:45.289097 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.289080    2581 scope.go:117] "RemoveContainer" containerID="a3e124e7a3bdf0b35e42172bd246ac56cf6a4ade59bf6b48a3230c88f6a401ec"
Sep 14 13:18:45.289359 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:45.289340    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3e124e7a3bdf0b35e42172bd246ac56cf6a4ade59bf6b48a3230c88f6a401ec\": container with ID starting with a3e124e7a3bdf0b35e42172bd246ac56cf6a4ade59bf6b48a3230c88f6a401ec not found: ID does not exist" containerID="a3e124e7a3bdf0b35e42172bd246ac56cf6a4ade59bf6b48a3230c88f6a401ec"
Sep 14 13:18:45.289416 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.289365    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3e124e7a3bdf0b35e42172bd246ac56cf6a4ade59bf6b48a3230c88f6a401ec"} err="failed to get container status \"a3e124e7a3bdf0b35e42172bd246ac56cf6a4ade59bf6b48a3230c88f6a401ec\": rpc error: code = NotFound desc = could not find container \"a3e124e7a3bdf0b35e42172bd246ac56cf6a4ade59bf6b48a3230c88f6a401ec\": container with ID starting with a3e124e7a3bdf0b35e42172bd246ac56cf6a4ade59bf6b48a3230c88f6a401ec not found: ID does not exist"
Sep 14 13:18:45.289416 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.289380    2581 scope.go:117] "RemoveContainer" containerID="c1c755b4f86a7b199361b610eefcea49a2650e8cc95a82be257ff0b8423dffd0"
Sep 14 13:18:45.294442 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.294419    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"]
Sep 14 13:18:45.297879 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.297859    2581 scope.go:117] "RemoveContainer" containerID="c1c755b4f86a7b199361b610eefcea49a2650e8cc95a82be257ff0b8423dffd0"
Sep 14 13:18:45.298200 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:45.298180    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1c755b4f86a7b199361b610eefcea49a2650e8cc95a82be257ff0b8423dffd0\": container with ID starting with c1c755b4f86a7b199361b610eefcea49a2650e8cc95a82be257ff0b8423dffd0 not found: ID does not exist" containerID="c1c755b4f86a7b199361b610eefcea49a2650e8cc95a82be257ff0b8423dffd0"
Sep 14 13:18:45.298279 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.298206    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1c755b4f86a7b199361b610eefcea49a2650e8cc95a82be257ff0b8423dffd0"} err="failed to get container status \"c1c755b4f86a7b199361b610eefcea49a2650e8cc95a82be257ff0b8423dffd0\": rpc error: code = NotFound desc = could not find container \"c1c755b4f86a7b199361b610eefcea49a2650e8cc95a82be257ff0b8423dffd0\": container with ID starting with c1c755b4f86a7b199361b610eefcea49a2650e8cc95a82be257ff0b8423dffd0 not found: ID does not exist"
Sep 14 13:18:45.299691 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.299669    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-cadd9c-6d6875cb5f-qs9x9"]
Sep 14 13:18:45.316792 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.316764    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"]
Sep 14 13:18:45.323466 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.323123    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-cadd9c-58bc54977c-vg89v"]
Sep 14 13:18:45.336969 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.336941    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sh5r\" (UniqueName: \"kubernetes.io/projected/e0714d67-3a61-4505-93d2-94c1046fa8a3-kube-api-access-5sh5r\") pod \"test-curl-7419-9e826f\" (UID: \"e0714d67-3a61-4505-93d2-94c1046fa8a3\") " pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:45.342104 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.342083    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"]
Sep 14 13:18:45.354492 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.354435    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-cadd9c-6b465f888b-w295q"]
Sep 14 13:18:45.438210 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.438175    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sh5r\" (UniqueName: \"kubernetes.io/projected/e0714d67-3a61-4505-93d2-94c1046fa8a3-kube-api-access-5sh5r\") pod \"test-curl-7419-9e826f\" (UID: \"e0714d67-3a61-4505-93d2-94c1046fa8a3\") " pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:45.452106 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.452075    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sh5r\" (UniqueName: \"kubernetes.io/projected/e0714d67-3a61-4505-93d2-94c1046fa8a3-kube-api-access-5sh5r\") pod \"test-curl-7419-9e826f\" (UID: \"e0714d67-3a61-4505-93d2-94c1046fa8a3\") " pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:45.541471 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.541437    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:45.642628 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.642552    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:45.815975 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:45.815952    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7419-9e826f"]
Sep 14 13:18:45.817736 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:18:45.817707    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0714d67_3a61_4505_93d2_94c1046fa8a3.slice/crio-a15590cdfea42c80df43e67f288be4f17c742ed1357cea27ff61eaa9ae167dff WatchSource:0}: Error finding container a15590cdfea42c80df43e67f288be4f17c742ed1357cea27ff61eaa9ae167dff: Status 404 returned error can't find the container with id a15590cdfea42c80df43e67f288be4f17c742ed1357cea27ff61eaa9ae167dff
Sep 14 13:18:46.278719 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:46.278689    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7419-9e826f" event={"ID":"e0714d67-3a61-4505-93d2-94c1046fa8a3","Type":"ContainerStarted","Data":"070573d53290b6860c2edd7c591b3fd588378e410c811970666e87f78abd28dd"}
Sep 14 13:18:46.279095 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:46.278728    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7419-9e826f" event={"ID":"e0714d67-3a61-4505-93d2-94c1046fa8a3","Type":"ContainerStarted","Data":"a15590cdfea42c80df43e67f288be4f17c742ed1357cea27ff61eaa9ae167dff"}
Sep 14 13:18:46.280719 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:46.280661    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7419-fac83c" podUID="762e051d-2522-4ee3-b91f-61292546bc75" containerName="test-curl-7419-fac83c" containerID="cri-o://30bcb6701ea7de38d3a520097ed7dd262c37f1dc63bfa4d22237b27715278190" gracePeriod=2
Sep 14 13:18:46.313367 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:46.313307    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7419-9e826f" podStartSLOduration=1.313282138 podStartE2EDuration="1.313282138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:18:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:18:46.31072964 +0000 UTC m=+1219.602063458" watchObservedRunningTime="2026-09-14 13:18:46.313282138 +0000 UTC m=+1219.604615959"
Sep 14 13:18:46.469769 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:46.469737    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-e267b2"
Sep 14 13:18:46.650439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:46.650402    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tqqz\" (UniqueName: \"kubernetes.io/projected/b30081d9-0975-40f7-a79c-d77e04096d77-kube-api-access-2tqqz\") pod \"b30081d9-0975-40f7-a79c-d77e04096d77\" (UID: \"b30081d9-0975-40f7-a79c-d77e04096d77\") "
Sep 14 13:18:46.652581 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:46.652546    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b30081d9-0975-40f7-a79c-d77e04096d77-kube-api-access-2tqqz" (OuterVolumeSpecName: "kube-api-access-2tqqz") pod "b30081d9-0975-40f7-a79c-d77e04096d77" (UID: "b30081d9-0975-40f7-a79c-d77e04096d77"). InnerVolumeSpecName "kube-api-access-2tqqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:18:46.752166 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:46.752126    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tqqz\" (UniqueName: \"kubernetes.io/projected/b30081d9-0975-40f7-a79c-d77e04096d77-kube-api-access-2tqqz\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:46.818047 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:46.818022    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:47.052998 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.052964    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7419-9e826f"]
Sep 14 13:18:47.064439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.064363    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7419-9e826f"]
Sep 14 13:18:47.203651 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.203613    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7419-c63d02"]
Sep 14 13:18:47.204207 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.204188    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0714d67-3a61-4505-93d2-94c1046fa8a3" containerName="test-curl-7419-9e826f"
Sep 14 13:18:47.204290 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.204211    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0714d67-3a61-4505-93d2-94c1046fa8a3" containerName="test-curl-7419-9e826f"
Sep 14 13:18:47.204337 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.204292    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0714d67-3a61-4505-93d2-94c1046fa8a3" containerName="test-curl-7419-9e826f"
Sep 14 13:18:47.213013 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.212136    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-c63d02"
Sep 14 13:18:47.219690 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.219115    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7419-c63d02"]
Sep 14 13:18:47.247794 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.247762    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23719951-7853-459f-afed-b3ba7c2b8660" path="/var/lib/kubelet/pods/23719951-7853-459f-afed-b3ba7c2b8660/volumes"
Sep 14 13:18:47.248175 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.248122    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af5c1011-5523-466c-b844-787991bce9db" path="/var/lib/kubelet/pods/af5c1011-5523-466c-b844-787991bce9db/volumes"
Sep 14 13:18:47.248572 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.248545    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b30081d9-0975-40f7-a79c-d77e04096d77" path="/var/lib/kubelet/pods/b30081d9-0975-40f7-a79c-d77e04096d77/volumes"
Sep 14 13:18:47.249095 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.249074    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba5f701e-637f-44b4-8b4a-090d42179f7e" path="/var/lib/kubelet/pods/ba5f701e-637f-44b4-8b4a-090d42179f7e/volumes"
Sep 14 13:18:47.286473 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.286442    2581 generic.go:356] "Generic (PLEG): container finished" podID="b30081d9-0975-40f7-a79c-d77e04096d77" containerID="be9b4770a3f9d075e7367cb7ddb9d369c6636c01587c987e992cecf06cc3908d" exitCode=137
Sep 14 13:18:47.286953 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.286510    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-e267b2"
Sep 14 13:18:47.286953 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.286570    2581 scope.go:117] "RemoveContainer" containerID="be9b4770a3f9d075e7367cb7ddb9d369c6636c01587c987e992cecf06cc3908d"
Sep 14 13:18:47.286953 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:47.286747    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7419-9e826f" container="test-curl-7419-9e826f"
Sep 14 13:18:47.289254 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:47.289230    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-e267b2\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="b30081d9-0975-40f7-a79c-d77e04096d77" pod="opendatahub/test-curl-7419-e267b2"
Sep 14 13:18:47.292888 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:47.292860    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-9e826f\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="e0714d67-3a61-4505-93d2-94c1046fa8a3" pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:47.295915 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.295892    2581 scope.go:117] "RemoveContainer" containerID="be9b4770a3f9d075e7367cb7ddb9d369c6636c01587c987e992cecf06cc3908d"
Sep 14 13:18:47.296285 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:47.296263    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be9b4770a3f9d075e7367cb7ddb9d369c6636c01587c987e992cecf06cc3908d\": container with ID starting with be9b4770a3f9d075e7367cb7ddb9d369c6636c01587c987e992cecf06cc3908d not found: ID does not exist" containerID="be9b4770a3f9d075e7367cb7ddb9d369c6636c01587c987e992cecf06cc3908d"
Sep 14 13:18:47.296368 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.296290    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be9b4770a3f9d075e7367cb7ddb9d369c6636c01587c987e992cecf06cc3908d"} err="failed to get container status \"be9b4770a3f9d075e7367cb7ddb9d369c6636c01587c987e992cecf06cc3908d\": rpc error: code = NotFound desc = could not find container \"be9b4770a3f9d075e7367cb7ddb9d369c6636c01587c987e992cecf06cc3908d\": container with ID starting with be9b4770a3f9d075e7367cb7ddb9d369c6636c01587c987e992cecf06cc3908d not found: ID does not exist"
Sep 14 13:18:47.298608 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:47.298582    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-e267b2\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="b30081d9-0975-40f7-a79c-d77e04096d77" pod="opendatahub/test-curl-7419-e267b2"
Sep 14 13:18:47.301334 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:47.301308    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-9e826f\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="e0714d67-3a61-4505-93d2-94c1046fa8a3" pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:47.357590 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.357554    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7kwz\" (UniqueName: \"kubernetes.io/projected/b1342146-d3d6-4236-918d-5cbf9c33c92b-kube-api-access-f7kwz\") pod \"test-curl-7419-c63d02\" (UID: \"b1342146-d3d6-4236-918d-5cbf9c33c92b\") " pod="opendatahub/test-curl-7419-c63d02"
Sep 14 13:18:47.459001 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.458965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7kwz\" (UniqueName: \"kubernetes.io/projected/b1342146-d3d6-4236-918d-5cbf9c33c92b-kube-api-access-f7kwz\") pod \"test-curl-7419-c63d02\" (UID: \"b1342146-d3d6-4236-918d-5cbf9c33c92b\") " pod="opendatahub/test-curl-7419-c63d02"
Sep 14 13:18:47.467563 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.467516    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7kwz\" (UniqueName: \"kubernetes.io/projected/b1342146-d3d6-4236-918d-5cbf9c33c92b-kube-api-access-f7kwz\") pod \"test-curl-7419-c63d02\" (UID: \"b1342146-d3d6-4236-918d-5cbf9c33c92b\") " pod="opendatahub/test-curl-7419-c63d02"
Sep 14 13:18:47.524883 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.524836    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-c63d02"
Sep 14 13:18:47.723644 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.723516    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:47.816270 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:47.816244    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7419-c63d02"]
Sep 14 13:18:47.818247 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:18:47.818220    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1342146_d3d6_4236_918d_5cbf9c33c92b.slice/crio-99231e46b9360de2ef86c27cc5d20df115cce0c30bc58de17cb6db801d02bc75 WatchSource:0}: Error finding container 99231e46b9360de2ef86c27cc5d20df115cce0c30bc58de17cb6db801d02bc75: Status 404 returned error can't find the container with id 99231e46b9360de2ef86c27cc5d20df115cce0c30bc58de17cb6db801d02bc75
Sep 14 13:18:48.293320 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:48.293292    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7419-c63d02" event={"ID":"b1342146-d3d6-4236-918d-5cbf9c33c92b","Type":"ContainerStarted","Data":"23114f8cb8a01bd11681b6a407cc8358dbb38899b61cc44e2ba589ba2fef430b"}
Sep 14 13:18:48.293700 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:48.293334    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7419-c63d02" event={"ID":"b1342146-d3d6-4236-918d-5cbf9c33c92b","Type":"ContainerStarted","Data":"99231e46b9360de2ef86c27cc5d20df115cce0c30bc58de17cb6db801d02bc75"}
Sep 14 13:18:48.294302 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:48.294278    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7419-9e826f" podUID="e0714d67-3a61-4505-93d2-94c1046fa8a3" containerName="test-curl-7419-9e826f" containerID="cri-o://070573d53290b6860c2edd7c591b3fd588378e410c811970666e87f78abd28dd" gracePeriod=2
Sep 14 13:18:48.295701 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:48.295678    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-9e826f\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="e0714d67-3a61-4505-93d2-94c1046fa8a3" pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:48.312701 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:48.312655    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7419-c63d02" podStartSLOduration=1.3126403039999999 podStartE2EDuration="1.312640304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:18:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:18:48.310993587 +0000 UTC m=+1221.602327404" watchObservedRunningTime="2026-09-14 13:18:48.312640304 +0000 UTC m=+1221.603974120"
Sep 14 13:18:48.313072 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:48.313049    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-e267b2\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="b30081d9-0975-40f7-a79c-d77e04096d77" pod="opendatahub/test-curl-7419-e267b2"
Sep 14 13:18:48.315016 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:48.314989    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-e267b2\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="b30081d9-0975-40f7-a79c-d77e04096d77" pod="opendatahub/test-curl-7419-e267b2"
Sep 14 13:18:48.316922 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:48.316900    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-9e826f\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="e0714d67-3a61-4505-93d2-94c1046fa8a3" pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:48.432974 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:48.432949    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-fac83c"
Sep 14 13:18:48.435641 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:48.435617    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-9e826f\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="e0714d67-3a61-4505-93d2-94c1046fa8a3" pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:48.437890 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:48.437866    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-fac83c\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="762e051d-2522-4ee3-b91f-61292546bc75" pod="opendatahub/test-curl-7419-fac83c"
Sep 14 13:18:48.439932 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:48.439913    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-e267b2\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="b30081d9-0975-40f7-a79c-d77e04096d77" pod="opendatahub/test-curl-7419-e267b2"
Sep 14 13:18:48.544737 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:48.544658    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:48.570988 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:48.570947    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6pjw\" (UniqueName: \"kubernetes.io/projected/762e051d-2522-4ee3-b91f-61292546bc75-kube-api-access-k6pjw\") pod \"762e051d-2522-4ee3-b91f-61292546bc75\" (UID: \"762e051d-2522-4ee3-b91f-61292546bc75\") "
Sep 14 13:18:48.573029 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:48.573002    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/762e051d-2522-4ee3-b91f-61292546bc75-kube-api-access-k6pjw" (OuterVolumeSpecName: "kube-api-access-k6pjw") pod "762e051d-2522-4ee3-b91f-61292546bc75" (UID: "762e051d-2522-4ee3-b91f-61292546bc75"). InnerVolumeSpecName "kube-api-access-k6pjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:18:48.671930 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:48.671893    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6pjw\" (UniqueName: \"kubernetes.io/projected/762e051d-2522-4ee3-b91f-61292546bc75-kube-api-access-k6pjw\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:48.928863 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:48.928826    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7419-c63d02"]
Sep 14 13:18:48.945551 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:48.945501    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7419-c63d02"]
Sep 14 13:18:49.246131 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:49.246040    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="762e051d-2522-4ee3-b91f-61292546bc75" path="/var/lib/kubelet/pods/762e051d-2522-4ee3-b91f-61292546bc75/volumes"
Sep 14 13:18:49.301590 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:49.301550    2581 generic.go:356] "Generic (PLEG): container finished" podID="762e051d-2522-4ee3-b91f-61292546bc75" containerID="30bcb6701ea7de38d3a520097ed7dd262c37f1dc63bfa4d22237b27715278190" exitCode=137
Sep 14 13:18:49.301590 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:49.301593    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-fac83c"
Sep 14 13:18:49.302072 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:49.301648    2581 scope.go:117] "RemoveContainer" containerID="30bcb6701ea7de38d3a520097ed7dd262c37f1dc63bfa4d22237b27715278190"
Sep 14 13:18:49.302072 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:49.301814    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7419-c63d02" container="test-curl-7419-c63d02"
Sep 14 13:18:49.304418 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:49.304388    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-fac83c\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="762e051d-2522-4ee3-b91f-61292546bc75" pod="opendatahub/test-curl-7419-fac83c"
Sep 14 13:18:49.307395 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:49.307370    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-fac83c\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="762e051d-2522-4ee3-b91f-61292546bc75" pod="opendatahub/test-curl-7419-fac83c"
Sep 14 13:18:49.310481 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:49.310463    2581 scope.go:117] "RemoveContainer" containerID="30bcb6701ea7de38d3a520097ed7dd262c37f1dc63bfa4d22237b27715278190"
Sep 14 13:18:49.310802 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:49.310782    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30bcb6701ea7de38d3a520097ed7dd262c37f1dc63bfa4d22237b27715278190\": container with ID starting with 30bcb6701ea7de38d3a520097ed7dd262c37f1dc63bfa4d22237b27715278190 not found: ID does not exist" containerID="30bcb6701ea7de38d3a520097ed7dd262c37f1dc63bfa4d22237b27715278190"
Sep 14 13:18:49.310855 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:49.310815    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30bcb6701ea7de38d3a520097ed7dd262c37f1dc63bfa4d22237b27715278190"} err="failed to get container status \"30bcb6701ea7de38d3a520097ed7dd262c37f1dc63bfa4d22237b27715278190\": rpc error: code = NotFound desc = could not find container \"30bcb6701ea7de38d3a520097ed7dd262c37f1dc63bfa4d22237b27715278190\": container with ID starting with 30bcb6701ea7de38d3a520097ed7dd262c37f1dc63bfa4d22237b27715278190 not found: ID does not exist"
Sep 14 13:18:49.594801 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:49.594705    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:49.984213 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:49.984166    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:50.103214 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:50.103181    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:50.277315 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:50.277228    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:50.308258 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:50.308227    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7419-c63d02" podUID="b1342146-d3d6-4236-918d-5cbf9c33c92b" containerName="test-curl-7419-c63d02" containerID="cri-o://23114f8cb8a01bd11681b6a407cc8358dbb38899b61cc44e2ba589ba2fef430b" gracePeriod=2
Sep 14 13:18:50.310827 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:50.310802    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-fac83c\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="762e051d-2522-4ee3-b91f-61292546bc75" pod="opendatahub/test-curl-7419-fac83c"
Sep 14 13:18:50.473679 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:50.473654    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:50.476192 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:50.476159    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-9e826f\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="e0714d67-3a61-4505-93d2-94c1046fa8a3" pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:50.478442 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:50.478413    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-fac83c\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="762e051d-2522-4ee3-b91f-61292546bc75" pod="opendatahub/test-curl-7419-fac83c"
Sep 14 13:18:50.486832 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:50.486809    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sh5r\" (UniqueName: \"kubernetes.io/projected/e0714d67-3a61-4505-93d2-94c1046fa8a3-kube-api-access-5sh5r\") pod \"e0714d67-3a61-4505-93d2-94c1046fa8a3\" (UID: \"e0714d67-3a61-4505-93d2-94c1046fa8a3\") "
Sep 14 13:18:50.489049 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:50.489019    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0714d67-3a61-4505-93d2-94c1046fa8a3-kube-api-access-5sh5r" (OuterVolumeSpecName: "kube-api-access-5sh5r") pod "e0714d67-3a61-4505-93d2-94c1046fa8a3" (UID: "e0714d67-3a61-4505-93d2-94c1046fa8a3"). InnerVolumeSpecName "kube-api-access-5sh5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:18:50.587975 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:50.587881    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sh5r\" (UniqueName: \"kubernetes.io/projected/e0714d67-3a61-4505-93d2-94c1046fa8a3-kube-api-access-5sh5r\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:51.246022 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:51.245989    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0714d67-3a61-4505-93d2-94c1046fa8a3" path="/var/lib/kubelet/pods/e0714d67-3a61-4505-93d2-94c1046fa8a3/volumes"
Sep 14 13:18:51.316000 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:51.315960    2581 generic.go:356] "Generic (PLEG): container finished" podID="e0714d67-3a61-4505-93d2-94c1046fa8a3" containerID="070573d53290b6860c2edd7c591b3fd588378e410c811970666e87f78abd28dd" exitCode=137
Sep 14 13:18:51.316473 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:51.316018    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:51.316473 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:51.316059    2581 scope.go:117] "RemoveContainer" containerID="070573d53290b6860c2edd7c591b3fd588378e410c811970666e87f78abd28dd"
Sep 14 13:18:51.318829 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:51.318803    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-9e826f\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="e0714d67-3a61-4505-93d2-94c1046fa8a3" pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:51.321104 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:51.321076    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-9e826f\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="e0714d67-3a61-4505-93d2-94c1046fa8a3" pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:51.325080 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:51.325059    2581 scope.go:117] "RemoveContainer" containerID="070573d53290b6860c2edd7c591b3fd588378e410c811970666e87f78abd28dd"
Sep 14 13:18:51.325362 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:51.325342    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"070573d53290b6860c2edd7c591b3fd588378e410c811970666e87f78abd28dd\": container with ID starting with 070573d53290b6860c2edd7c591b3fd588378e410c811970666e87f78abd28dd not found: ID does not exist" containerID="070573d53290b6860c2edd7c591b3fd588378e410c811970666e87f78abd28dd"
Sep 14 13:18:51.325411 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:51.325371    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"070573d53290b6860c2edd7c591b3fd588378e410c811970666e87f78abd28dd"} err="failed to get container status \"070573d53290b6860c2edd7c591b3fd588378e410c811970666e87f78abd28dd\": rpc error: code = NotFound desc = could not find container \"070573d53290b6860c2edd7c591b3fd588378e410c811970666e87f78abd28dd\": container with ID starting with 070573d53290b6860c2edd7c591b3fd588378e410c811970666e87f78abd28dd not found: ID does not exist"
Sep 14 13:18:52.028436 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:52.028408    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:52.494589 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:52.494563    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-c63d02"
Sep 14 13:18:52.497167 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:52.497139    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7419-9e826f\" is forbidden: User \"system:node:ip-10-0-135-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-35.ec2.internal' and this object" podUID="e0714d67-3a61-4505-93d2-94c1046fa8a3" pod="opendatahub/test-curl-7419-9e826f"
Sep 14 13:18:52.502693 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:52.502666    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7kwz\" (UniqueName: \"kubernetes.io/projected/b1342146-d3d6-4236-918d-5cbf9c33c92b-kube-api-access-f7kwz\") pod \"b1342146-d3d6-4236-918d-5cbf9c33c92b\" (UID: \"b1342146-d3d6-4236-918d-5cbf9c33c92b\") "
Sep 14 13:18:52.504646 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:52.504618    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1342146-d3d6-4236-918d-5cbf9c33c92b-kube-api-access-f7kwz" (OuterVolumeSpecName: "kube-api-access-f7kwz") pod "b1342146-d3d6-4236-918d-5cbf9c33c92b" (UID: "b1342146-d3d6-4236-918d-5cbf9c33c92b"). InnerVolumeSpecName "kube-api-access-f7kwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:18:52.604405 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:52.604369    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7kwz\" (UniqueName: \"kubernetes.io/projected/b1342146-d3d6-4236-918d-5cbf9c33c92b-kube-api-access-f7kwz\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:18:52.719627 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:52.719591    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:52.815900 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:52.815835    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:53.245458 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:53.245426    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1342146-d3d6-4236-918d-5cbf9c33c92b" path="/var/lib/kubelet/pods/b1342146-d3d6-4236-918d-5cbf9c33c92b/volumes"
Sep 14 13:18:53.328027 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:53.327996    2581 generic.go:356] "Generic (PLEG): container finished" podID="b1342146-d3d6-4236-918d-5cbf9c33c92b" containerID="23114f8cb8a01bd11681b6a407cc8358dbb38899b61cc44e2ba589ba2fef430b" exitCode=137
Sep 14 13:18:53.328238 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:53.328038    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7419-c63d02"
Sep 14 13:18:53.328238 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:53.328099    2581 scope.go:117] "RemoveContainer" containerID="23114f8cb8a01bd11681b6a407cc8358dbb38899b61cc44e2ba589ba2fef430b"
Sep 14 13:18:53.336525 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:53.336505    2581 scope.go:117] "RemoveContainer" containerID="23114f8cb8a01bd11681b6a407cc8358dbb38899b61cc44e2ba589ba2fef430b"
Sep 14 13:18:53.336814 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:53.336798    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23114f8cb8a01bd11681b6a407cc8358dbb38899b61cc44e2ba589ba2fef430b\": container with ID starting with 23114f8cb8a01bd11681b6a407cc8358dbb38899b61cc44e2ba589ba2fef430b not found: ID does not exist" containerID="23114f8cb8a01bd11681b6a407cc8358dbb38899b61cc44e2ba589ba2fef430b"
Sep 14 13:18:53.336861 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:53.336822    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23114f8cb8a01bd11681b6a407cc8358dbb38899b61cc44e2ba589ba2fef430b"} err="failed to get container status \"23114f8cb8a01bd11681b6a407cc8358dbb38899b61cc44e2ba589ba2fef430b\": rpc error: code = NotFound desc = could not find container \"23114f8cb8a01bd11681b6a407cc8358dbb38899b61cc44e2ba589ba2fef430b\": container with ID starting with 23114f8cb8a01bd11681b6a407cc8358dbb38899b61cc44e2ba589ba2fef430b not found: ID does not exist"
Sep 14 13:18:53.556472 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:53.556389    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:54.424198 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:54.424166    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:55.105721 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:55.105681    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:55.923640 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:55.923609    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"]
Sep 14 13:18:55.924163 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:55.924138    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1342146-d3d6-4236-918d-5cbf9c33c92b" containerName="test-curl-7419-c63d02"
Sep 14 13:18:55.924163 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:55.924164    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1342146-d3d6-4236-918d-5cbf9c33c92b" containerName="test-curl-7419-c63d02"
Sep 14 13:18:55.924345 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:55.924281    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1342146-d3d6-4236-918d-5cbf9c33c92b" containerName="test-curl-7419-c63d02"
Sep 14 13:18:55.927437 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:55.927405    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:55.930086 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:55.930060    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-336ccc18-serving-cert\""
Sep 14 13:18:55.941238 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:55.941211    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"]
Sep 14 13:18:55.954782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:55.954760    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:56.033129 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.033087    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-maas-api-tls\") pod \"maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:56.033303 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.033162    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdd89\" (UniqueName: \"kubernetes.io/projected/2f7cd24f-017b-4572-9897-331a916d5b2d-kube-api-access-vdd89\") pod \"maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:56.033303 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.033221    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-metrics-certs\") pod \"maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:56.033395 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.033333    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f7cd24f-017b-4572-9897-331a916d5b2d-openshift-service-ca\") pod \"maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:56.056784 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.056757    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:56.134438 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.134399    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-metrics-certs\") pod \"maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:56.134908 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.134585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f7cd24f-017b-4572-9897-331a916d5b2d-openshift-service-ca\") pod \"maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:56.134908 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.134654    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-maas-api-tls\") pod \"maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:56.134908 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.134712    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdd89\" (UniqueName: \"kubernetes.io/projected/2f7cd24f-017b-4572-9897-331a916d5b2d-kube-api-access-vdd89\") pod \"maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:56.134908 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:56.134783    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-serving-cert: secret "maas-api-e2e-mt-336ccc18-serving-cert" not found
Sep 14 13:18:56.134908 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:56.134862    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-maas-api-tls podName:2f7cd24f-017b-4572-9897-331a916d5b2d nodeName:}" failed. No retries permitted until 2026-09-14 13:18:56.634842343 +0000 UTC m=+1229.926176143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-maas-api-tls") pod "maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm" (UID: "2f7cd24f-017b-4572-9897-331a916d5b2d") : secret "maas-api-e2e-mt-336ccc18-serving-cert" not found
Sep 14 13:18:56.135337 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.135314    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f7cd24f-017b-4572-9897-331a916d5b2d-openshift-service-ca\") pod \"maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:56.136993 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.136976    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-metrics-certs\") pod \"maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:56.147794 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.147769    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdd89\" (UniqueName: \"kubernetes.io/projected/2f7cd24f-017b-4572-9897-331a916d5b2d-kube-api-access-vdd89\") pod \"maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:56.431678 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.431631    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:56.533185 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.533101    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:56.541139 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.541106    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"]
Sep 14 13:18:56.545301 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.545279    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"
Sep 14 13:18:56.547881 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.547857    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-336ccc18\""
Sep 14 13:18:56.547995 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.547925    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-336ccc18-dockercfg-pcvtn\""
Sep 14 13:18:56.561065 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.561031    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"]
Sep 14 13:18:56.574278 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.574246    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"]
Sep 14 13:18:56.577800 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.577765    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"
Sep 14 13:18:56.590926 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.590900    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"]
Sep 14 13:18:56.639147 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.639116    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7e1829e-3ae5-4f5c-9cf9-56642927b55d-plugins-config-volume\") pod \"payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv\" (UID: \"f7e1829e-3ae5-4f5c-9cf9-56642927b55d\") " pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"
Sep 14 13:18:56.639304 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.639153    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp6gw\" (UniqueName: \"kubernetes.io/projected/bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b-kube-api-access-xp6gw\") pod \"payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq\" (UID: \"bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"
Sep 14 13:18:56.639304 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.639186    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-maas-api-tls\") pod \"maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:56.639304 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.639283    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq\" (UID: \"bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"
Sep 14 13:18:56.639458 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:56.639329    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-serving-cert: secret "maas-api-e2e-mt-336ccc18-serving-cert" not found
Sep 14 13:18:56.639458 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.639351    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7qg6\" (UniqueName: \"kubernetes.io/projected/f7e1829e-3ae5-4f5c-9cf9-56642927b55d-kube-api-access-w7qg6\") pod \"payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv\" (UID: \"f7e1829e-3ae5-4f5c-9cf9-56642927b55d\") " pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"
Sep 14 13:18:56.639458 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:18:56.639382    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-maas-api-tls podName:2f7cd24f-017b-4572-9897-331a916d5b2d nodeName:}" failed. No retries permitted until 2026-09-14 13:18:57.639362902 +0000 UTC m=+1230.930696711 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-maas-api-tls") pod "maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm" (UID: "2f7cd24f-017b-4572-9897-331a916d5b2d") : secret "maas-api-e2e-mt-336ccc18-serving-cert" not found
Sep 14 13:18:56.740612 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.740494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7e1829e-3ae5-4f5c-9cf9-56642927b55d-plugins-config-volume\") pod \"payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv\" (UID: \"f7e1829e-3ae5-4f5c-9cf9-56642927b55d\") " pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"
Sep 14 13:18:56.740612 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.740548    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp6gw\" (UniqueName: \"kubernetes.io/projected/bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b-kube-api-access-xp6gw\") pod \"payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq\" (UID: \"bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"
Sep 14 13:18:56.740612 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.740602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq\" (UID: \"bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"
Sep 14 13:18:56.740903 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.740636    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7qg6\" (UniqueName: \"kubernetes.io/projected/f7e1829e-3ae5-4f5c-9cf9-56642927b55d-kube-api-access-w7qg6\") pod \"payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv\" (UID: \"f7e1829e-3ae5-4f5c-9cf9-56642927b55d\") " pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"
Sep 14 13:18:56.741239 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.741217    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7e1829e-3ae5-4f5c-9cf9-56642927b55d-plugins-config-volume\") pod \"payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv\" (UID: \"f7e1829e-3ae5-4f5c-9cf9-56642927b55d\") " pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"
Sep 14 13:18:56.741341 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.741323    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq\" (UID: \"bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"
Sep 14 13:18:56.749303 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.749278    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7qg6\" (UniqueName: \"kubernetes.io/projected/f7e1829e-3ae5-4f5c-9cf9-56642927b55d-kube-api-access-w7qg6\") pod \"payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv\" (UID: \"f7e1829e-3ae5-4f5c-9cf9-56642927b55d\") " pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"
Sep 14 13:18:56.749393 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.749338    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp6gw\" (UniqueName: \"kubernetes.io/projected/bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b-kube-api-access-xp6gw\") pod \"payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq\" (UID: \"bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"
Sep 14 13:18:56.756446 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.756413    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:18:56.865698 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.865654    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"
Sep 14 13:18:56.888610 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:56.888581    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"
Sep 14 13:18:57.147641 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.147617    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"]
Sep 14 13:18:57.150011 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:18:57.149982    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcfbd3eb_e27b_466b_903f_8f22ef6b1c8b.slice/crio-fb69a60673f92a45c0b10a5b6dbf9fbdd14230d855f030c0244ee32783dec39c WatchSource:0}: Error finding container fb69a60673f92a45c0b10a5b6dbf9fbdd14230d855f030c0244ee32783dec39c: Status 404 returned error can't find the container with id fb69a60673f92a45c0b10a5b6dbf9fbdd14230d855f030c0244ee32783dec39c
Sep 14 13:18:57.157751 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.157728    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:57.182131 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.182110    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"]
Sep 14 13:18:57.192734 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:18:57.192708    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e1829e_3ae5_4f5c_9cf9_56642927b55d.slice/crio-09338b7fa2e55dfe9bbc0fbd4ec4de2c0060ce53f3a36a096d3a8c6d26a85ad6 WatchSource:0}: Error finding container 09338b7fa2e55dfe9bbc0fbd4ec4de2c0060ce53f3a36a096d3a8c6d26a85ad6: Status 404 returned error can't find the container with id 09338b7fa2e55dfe9bbc0fbd4ec4de2c0060ce53f3a36a096d3a8c6d26a85ad6
Sep 14 13:18:57.351564 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.351509    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv" event={"ID":"f7e1829e-3ae5-4f5c-9cf9-56642927b55d","Type":"ContainerStarted","Data":"e6a0521741e4cd9979c16cd86cf1cf58dbb5c2a1345be093efb4ffaf5fbe09e9"}
Sep 14 13:18:57.351753 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.351590    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"
Sep 14 13:18:57.351753 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.351616    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv" event={"ID":"f7e1829e-3ae5-4f5c-9cf9-56642927b55d","Type":"ContainerStarted","Data":"09338b7fa2e55dfe9bbc0fbd4ec4de2c0060ce53f3a36a096d3a8c6d26a85ad6"}
Sep 14 13:18:57.353106 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.353074    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq" event={"ID":"bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b","Type":"ContainerStarted","Data":"936b00ca0c25b3870c9d4e484f544ef47b65b2c8f1775dbee12feb6bf09608b5"}
Sep 14 13:18:57.353236 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.353117    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq" event={"ID":"bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b","Type":"ContainerStarted","Data":"fb69a60673f92a45c0b10a5b6dbf9fbdd14230d855f030c0244ee32783dec39c"}
Sep 14 13:18:57.353236 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.353162    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"
Sep 14 13:18:57.369205 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.369152    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv" podStartSLOduration=1.369136461 podStartE2EDuration="1.369136461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:18:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:18:57.367598022 +0000 UTC m=+1230.658931842" watchObservedRunningTime="2026-09-14 13:18:57.369136461 +0000 UTC m=+1230.660470278"
Sep 14 13:18:57.382934 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.382883    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq" podStartSLOduration=1.382872454 podStartE2EDuration="1.382872454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:18:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:18:57.382174942 +0000 UTC m=+1230.673508772" watchObservedRunningTime="2026-09-14 13:18:57.382872454 +0000 UTC m=+1230.674206291"
Sep 14 13:18:57.649752 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.649715    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-maas-api-tls\") pod \"maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:57.652782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.652749    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-maas-api-tls\") pod \"maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:57.737752 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.737719    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:57.777157 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:57.777120    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:58.023325 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:58.023239    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"]
Sep 14 13:18:58.027388 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:18:58.027357    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f7cd24f_017b_4572_9897_331a916d5b2d.slice/crio-48fe0f799ed240b309314dd81867f8a62fc818943c14bc722f42b117cfb65016 WatchSource:0}: Error finding container 48fe0f799ed240b309314dd81867f8a62fc818943c14bc722f42b117cfb65016: Status 404 returned error can't find the container with id 48fe0f799ed240b309314dd81867f8a62fc818943c14bc722f42b117cfb65016
Sep 14 13:18:58.047561 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:58.047512    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:58.359428 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:58.359392    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm" event={"ID":"2f7cd24f-017b-4572-9897-331a916d5b2d","Type":"ContainerStarted","Data":"48fe0f799ed240b309314dd81867f8a62fc818943c14bc722f42b117cfb65016"}
Sep 14 13:18:58.642714 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:58.642635    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:58.992472 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:58.992381    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:59.145092 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:59.145060    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:59.365766 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:59.365730    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm" event={"ID":"2f7cd24f-017b-4572-9897-331a916d5b2d","Type":"ContainerStarted","Data":"84f194357ed64b4575350b427a9ea17c7d8531c5d414371e1e2ced1d8877a7e7"}
Sep 14 13:18:59.366252 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:59.365854    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:18:59.387525 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:18:59.387470    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm" podStartSLOduration=3.866554318 podStartE2EDuration="4.387453325s" totalImagesPullingTime="520.899007ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:18:59.386307993 +0000 UTC m=+1232.677641821" watchObservedRunningTime="2026-09-14 13:18:59.387453325 +0000 UTC m=+1232.678787143"
Sep 14 13:19:01.363341 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:01.363313    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:02.088596 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:02.088562    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:02.234650 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:02.234614    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:02.339750 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:02.339653    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:03.052452 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:03.052404    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:03.886685 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:03.886656    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:04.683910 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:04.683874    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:05.377101 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:05.377073    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:19:05.468471 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:05.468440    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:06.331026 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.330994    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"]
Sep 14 13:19:06.335081 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.335057    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:06.337925 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.337892    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-14f3af-serving-cert\""
Sep 14 13:19:06.357696 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.357671    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"]
Sep 14 13:19:06.438986 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.438955    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-872xk\" (UniqueName: \"kubernetes.io/projected/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-kube-api-access-872xk\") pod \"maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:06.438986 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.438986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:06.439218 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.439081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:06.439218 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.439110    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:06.540389 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.540346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-872xk\" (UniqueName: \"kubernetes.io/projected/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-kube-api-access-872xk\") pod \"maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:06.540389 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.540383    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:06.540680 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.540453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:06.540680 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.540495    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:06.541219 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.541200    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:06.542868 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.542847    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:06.543032 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.543011    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:06.553561 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.553512    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-872xk\" (UniqueName: \"kubernetes.io/projected/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-kube-api-access-872xk\") pod \"maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:06.646576 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.646516    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:06.930411 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.930387    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"]
Sep 14 13:19:06.932697 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:19:06.932669    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e34f45e_5dca_48cb_8f45_9a9d84be39a6.slice/crio-cbc3b00b4ffcb2b3e6f59b0a35eaa206c11be6b11a3cd0e66c67af3969bca7e0 WatchSource:0}: Error finding container cbc3b00b4ffcb2b3e6f59b0a35eaa206c11be6b11a3cd0e66c67af3969bca7e0: Status 404 returned error can't find the container with id cbc3b00b4ffcb2b3e6f59b0a35eaa206c11be6b11a3cd0e66c67af3969bca7e0
Sep 14 13:19:06.960414 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:06.960375    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:07.066811 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.066785    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:07.203732 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.203657    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"]
Sep 14 13:19:07.208420 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.208402    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"
Sep 14 13:19:07.214737 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.214714    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-14f3af\""
Sep 14 13:19:07.216093 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.216072    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-14f3af-dockercfg-5tndg\""
Sep 14 13:19:07.227956 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.227922    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"]
Sep 14 13:19:07.247148 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.247111    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z78h7\" (UniqueName: \"kubernetes.io/projected/d6073121-67f4-4240-827f-1060d208c62c-kube-api-access-z78h7\") pod \"payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn\" (UID: \"d6073121-67f4-4240-827f-1060d208c62c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"
Sep 14 13:19:07.247320 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.247161    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6073121-67f4-4240-827f-1060d208c62c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn\" (UID: \"d6073121-67f4-4240-827f-1060d208c62c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"
Sep 14 13:19:07.255157 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.255126    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:07.348811 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.348772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z78h7\" (UniqueName: \"kubernetes.io/projected/d6073121-67f4-4240-827f-1060d208c62c-kube-api-access-z78h7\") pod \"payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn\" (UID: \"d6073121-67f4-4240-827f-1060d208c62c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"
Sep 14 13:19:07.349317 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.348844    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6073121-67f4-4240-827f-1060d208c62c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn\" (UID: \"d6073121-67f4-4240-827f-1060d208c62c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"
Sep 14 13:19:07.349816 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.349779    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6073121-67f4-4240-827f-1060d208c62c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn\" (UID: \"d6073121-67f4-4240-827f-1060d208c62c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"
Sep 14 13:19:07.365909 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.365880    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z78h7\" (UniqueName: \"kubernetes.io/projected/d6073121-67f4-4240-827f-1060d208c62c-kube-api-access-z78h7\") pod \"payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn\" (UID: \"d6073121-67f4-4240-827f-1060d208c62c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"
Sep 14 13:19:07.416341 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.416307    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7" event={"ID":"3e34f45e-5dca-48cb-8f45-9a9d84be39a6","Type":"ContainerStarted","Data":"cbc3b00b4ffcb2b3e6f59b0a35eaa206c11be6b11a3cd0e66c67af3969bca7e0"}
Sep 14 13:19:07.519731 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.519645    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"
Sep 14 13:19:07.747331 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.747297    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:07.807999 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:07.807972    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"]
Sep 14 13:19:07.811257 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:19:07.811219    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6073121_67f4_4240_827f_1060d208c62c.slice/crio-a16b6476c9571cd048bd494b1522ae879aef32506e2eee753924dce99b7075c9 WatchSource:0}: Error finding container a16b6476c9571cd048bd494b1522ae879aef32506e2eee753924dce99b7075c9: Status 404 returned error can't find the container with id a16b6476c9571cd048bd494b1522ae879aef32506e2eee753924dce99b7075c9
Sep 14 13:19:08.361439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:08.361410    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"
Sep 14 13:19:08.361880 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:08.361859    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"
Sep 14 13:19:08.424761 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:08.424727    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7" event={"ID":"3e34f45e-5dca-48cb-8f45-9a9d84be39a6","Type":"ContainerStarted","Data":"277df7da569fb95615844d4fd9a296c1cea86ad3e5ffbf028f36d332f1367d94"}
Sep 14 13:19:08.424953 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:08.424845    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:08.426343 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:08.426319    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn" event={"ID":"d6073121-67f4-4240-827f-1060d208c62c","Type":"ContainerStarted","Data":"a88811301e746bb133932d9eff66df097a987cc99dc0344928f8280d32b4409c"}
Sep 14 13:19:08.426476 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:08.426359    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn" event={"ID":"d6073121-67f4-4240-827f-1060d208c62c","Type":"ContainerStarted","Data":"a16b6476c9571cd048bd494b1522ae879aef32506e2eee753924dce99b7075c9"}
Sep 14 13:19:08.426476 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:08.426430    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"
Sep 14 13:19:08.489048 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:08.488985    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7" podStartSLOduration=1.9008052979999999 podStartE2EDuration="2.488969751s" totalImagesPullingTime="588.164453ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:19:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:19:08.487374809 +0000 UTC m=+1241.778708628" watchObservedRunningTime="2026-09-14 13:19:08.488969751 +0000 UTC m=+1241.780303567"
Sep 14 13:19:08.517144 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:08.517094    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn" podStartSLOduration=1.517077428 podStartE2EDuration="1.517077428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:19:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:19:08.514893462 +0000 UTC m=+1241.806227279" watchObservedRunningTime="2026-09-14 13:19:08.517077428 +0000 UTC m=+1241.808411245"
Sep 14 13:19:09.165305 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:09.165276    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:10.514963 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:10.514929    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:14.437352 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:14.437318    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:19:16.813463 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:16.813430    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:18.283819 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:18.283786    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:19.252380 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:19.252347    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:19.434763 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:19.434686    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"
Sep 14 13:19:19.930557 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:19.930491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:20.573332 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:20.573303    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:21.348011 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:21.347982    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:22.012891 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:22.012861    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:22.822801 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:22.822750    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:22.939580 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:22.939543    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:23.285013 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:23.284973    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:19:23.753758 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:23.753725    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:24.660540 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:24.660496    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:25.664895 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:25.664855    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:26.433184 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:26.433151    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:27.141645 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:27.141617    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:27.243081 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:27.243045    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:27.558662 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:27.558571    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:27.898645 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:27.898609    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:27.978847 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:27.978816    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:28.202277 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.202188    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:28.672813 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.672781    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:28.912867 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.912837    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"]
Sep 14 13:19:28.918064 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.918045    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:28.921517 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.921492    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-4c0b93\"/\"noten-model-4c0b93-kserve-self-signed-certs\""
Sep 14 13:19:28.921517 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.921507    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-4c0b93\"/\"kube-root-ca.crt\""
Sep 14 13:19:28.921696 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.921507    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-4c0b93\"/\"default-dockercfg-nptgr\""
Sep 14 13:19:28.921696 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.921575    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-4c0b93\"/\"openshift-service-ca.crt\""
Sep 14 13:19:28.925887 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.925319    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"]
Sep 14 13:19:28.964923 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.964895    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsvwc\" (UniqueName: \"kubernetes.io/projected/f6bffe02-745d-4ce1-a344-c51ad20684dd-kube-api-access-jsvwc\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:28.965052 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.964936    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6bffe02-745d-4ce1-a344-c51ad20684dd-tls-certs\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:28.965052 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.964957    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-model-cache\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:28.965052 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.965008    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-dshm\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:28.965169 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.965066    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-tmp-dir\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:28.965169 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:28.965083    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-home\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:29.065900 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.065853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-dshm\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:29.066089 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.065949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-tmp-dir\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:29.066089 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.065970    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-home\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:29.066089 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.065999    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsvwc\" (UniqueName: \"kubernetes.io/projected/f6bffe02-745d-4ce1-a344-c51ad20684dd-kube-api-access-jsvwc\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:29.066203 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.066147    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6bffe02-745d-4ce1-a344-c51ad20684dd-tls-certs\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:29.066246 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.066196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-model-cache\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:29.066436 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.066409    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-tmp-dir\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:29.066436 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.066423    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-home\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:29.066616 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.066568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-model-cache\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:29.068264 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.068233    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-dshm\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:29.068583 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.068563    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6bffe02-745d-4ce1-a344-c51ad20684dd-tls-certs\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:29.074496 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.074470    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsvwc\" (UniqueName: \"kubernetes.io/projected/f6bffe02-745d-4ce1-a344-c51ad20684dd-kube-api-access-jsvwc\") pod \"noten-model-4c0b93-kserve-5bc76fff47-h48ts\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") " pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:29.204026 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.203946    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:29.227978 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.227941    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:29.458891 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.458802    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:29.513684 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.513657    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"]
Sep 14 13:19:29.515854 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:19:29.515830    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6bffe02_745d_4ce1_a344_c51ad20684dd.slice/crio-2047ee6c7806f95d036ca6579c82f49ff733542f5e87b376a84336fb7d263427 WatchSource:0}: Error finding container 2047ee6c7806f95d036ca6579c82f49ff733542f5e87b376a84336fb7d263427: Status 404 returned error can't find the container with id 2047ee6c7806f95d036ca6579c82f49ff733542f5e87b376a84336fb7d263427
Sep 14 13:19:29.544050 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:29.544018    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts" event={"ID":"f6bffe02-745d-4ce1-a344-c51ad20684dd","Type":"ContainerStarted","Data":"2047ee6c7806f95d036ca6579c82f49ff733542f5e87b376a84336fb7d263427"}
Sep 14 13:19:30.468827 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:30.468792    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:30.551678 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:30.551639    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts" event={"ID":"f6bffe02-745d-4ce1-a344-c51ad20684dd","Type":"ContainerStarted","Data":"563600f293b6c98ae22f3f1472659a7ffa2f82dad6f7614bc722c64fc543744f"}
Sep 14 13:19:30.570873 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:30.570822    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts" podStartSLOduration=2.570804641 podStartE2EDuration="2.570804641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:19:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:19:30.569091641 +0000 UTC m=+1263.860425457" watchObservedRunningTime="2026-09-14 13:19:30.570804641 +0000 UTC m=+1263.862138457"
Sep 14 13:19:31.129244 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:31.129193    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:31.218687 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:31.218653    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:31.443293 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:31.443210    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:32.257025 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:32.256977    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:33.141258 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:33.141226    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:33.814823 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:33.814789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:34.433525 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:34.433494    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:35.340170 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:35.340133    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:36.278065 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:36.278023    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:36.379878 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:36.379831    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:36.485490 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:36.485440    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:36.946516 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:36.946486    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:37.045282 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:37.045242    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"]
Sep 14 13:19:37.050222 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:37.050194    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"
Sep 14 13:19:37.053648 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:37.053623    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-eff5e8\""
Sep 14 13:19:37.062207 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:37.062176    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"]
Sep 14 13:19:37.063801 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:37.063768    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:37.144513 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:37.144477    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31459634-487d-4350-b79e-67dda73fc43f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv\" (UID: \"31459634-487d-4350-b79e-67dda73fc43f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"
Sep 14 13:19:37.144710 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:37.144575    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk9h2\" (UniqueName: \"kubernetes.io/projected/31459634-487d-4350-b79e-67dda73fc43f-kube-api-access-vk9h2\") pod \"payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv\" (UID: \"31459634-487d-4350-b79e-67dda73fc43f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"
Sep 14 13:19:37.245210 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:37.245127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31459634-487d-4350-b79e-67dda73fc43f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv\" (UID: \"31459634-487d-4350-b79e-67dda73fc43f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"
Sep 14 13:19:37.245210 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:37.245190    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk9h2\" (UniqueName: \"kubernetes.io/projected/31459634-487d-4350-b79e-67dda73fc43f-kube-api-access-vk9h2\") pod \"payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv\" (UID: \"31459634-487d-4350-b79e-67dda73fc43f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"
Sep 14 13:19:37.246060 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:37.246037    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31459634-487d-4350-b79e-67dda73fc43f-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv\" (UID: \"31459634-487d-4350-b79e-67dda73fc43f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"
Sep 14 13:19:37.256249 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:37.256224    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk9h2\" (UniqueName: \"kubernetes.io/projected/31459634-487d-4350-b79e-67dda73fc43f-kube-api-access-vk9h2\") pod \"payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv\" (UID: \"31459634-487d-4350-b79e-67dda73fc43f\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"
Sep 14 13:19:37.363433 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:37.363390    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"
Sep 14 13:19:37.650899 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:37.650804    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"]
Sep 14 13:19:37.653623 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:19:37.653590    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31459634_487d_4350_b79e_67dda73fc43f.slice/crio-e2dba9dcdde30f7fa475f2d660f08df991471dd6a50602dee9a1199f6474ca52 WatchSource:0}: Error finding container e2dba9dcdde30f7fa475f2d660f08df991471dd6a50602dee9a1199f6474ca52: Status 404 returned error can't find the container with id e2dba9dcdde30f7fa475f2d660f08df991471dd6a50602dee9a1199f6474ca52
Sep 14 13:19:38.599100 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:38.599059    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv" event={"ID":"31459634-487d-4350-b79e-67dda73fc43f","Type":"ContainerStarted","Data":"5c5027023f11f5b5c452d4addf8afa8591cdde65f3311c371babbc9dccfa6322"}
Sep 14 13:19:38.599313 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:38.599124    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"
Sep 14 13:19:38.599313 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:38.599144    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv" event={"ID":"31459634-487d-4350-b79e-67dda73fc43f","Type":"ContainerStarted","Data":"e2dba9dcdde30f7fa475f2d660f08df991471dd6a50602dee9a1199f6474ca52"}
Sep 14 13:19:38.622453 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:38.622402    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv" podStartSLOduration=1.622386724 podStartE2EDuration="1.622386724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:19:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:19:38.619056721 +0000 UTC m=+1271.910390554" watchObservedRunningTime="2026-09-14 13:19:38.622386724 +0000 UTC m=+1271.913720541"
Sep 14 13:19:39.228754 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:39.228713    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:39.229155 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:39.228967    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:39.246756 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:39.246723    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:39.617759 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:39.617728    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:19:49.608700 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:49.608659    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"
Sep 14 13:19:51.667797 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:51.667764    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:19:58.453666 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:58.453624    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:59.231738 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:59.231682    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:59.951763 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:19:59.951728    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:00.745645 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:00.745616    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:01.140630 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:01.140602    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:01.428068 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:01.427990    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:02.148147 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:02.148108    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:02.890762 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:02.890711    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:02.971161 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:02.971118    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:03.161061 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:03.160979    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:03.302783 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:03.302755    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:03.579130 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:03.579044    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:08.262679 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:08.262634    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"]
Sep 14 13:20:08.334896 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:08.334844    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:12.255467 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:12.255431    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:23.282781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.282714    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts" podUID="f6bffe02-745d-4ce1-a344-c51ad20684dd" containerName="main" containerID="cri-o://563600f293b6c98ae22f3f1472659a7ffa2f82dad6f7614bc722c64fc543744f" gracePeriod=45
Sep 14 13:20:23.543402 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.543338    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:20:23.601343 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.601311    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-home\") pod \"f6bffe02-745d-4ce1-a344-c51ad20684dd\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") "
Sep 14 13:20:23.601790 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.601381    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6bffe02-745d-4ce1-a344-c51ad20684dd-tls-certs\") pod \"f6bffe02-745d-4ce1-a344-c51ad20684dd\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") "
Sep 14 13:20:23.601790 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.601423    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-tmp-dir\") pod \"f6bffe02-745d-4ce1-a344-c51ad20684dd\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") "
Sep 14 13:20:23.601790 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.601458    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-dshm\") pod \"f6bffe02-745d-4ce1-a344-c51ad20684dd\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") "
Sep 14 13:20:23.601790 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.601480    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsvwc\" (UniqueName: \"kubernetes.io/projected/f6bffe02-745d-4ce1-a344-c51ad20684dd-kube-api-access-jsvwc\") pod \"f6bffe02-745d-4ce1-a344-c51ad20684dd\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") "
Sep 14 13:20:23.601790 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.601505    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-model-cache\") pod \"f6bffe02-745d-4ce1-a344-c51ad20684dd\" (UID: \"f6bffe02-745d-4ce1-a344-c51ad20684dd\") "
Sep 14 13:20:23.601790 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.601614    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-home" (OuterVolumeSpecName: "home") pod "f6bffe02-745d-4ce1-a344-c51ad20684dd" (UID: "f6bffe02-745d-4ce1-a344-c51ad20684dd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:20:23.601790 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.601726    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f6bffe02-745d-4ce1-a344-c51ad20684dd" (UID: "f6bffe02-745d-4ce1-a344-c51ad20684dd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:20:23.602208 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.601848    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-model-cache" (OuterVolumeSpecName: "model-cache") pod "f6bffe02-745d-4ce1-a344-c51ad20684dd" (UID: "f6bffe02-745d-4ce1-a344-c51ad20684dd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:20:23.602208 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.601872    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-home\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:20:23.602208 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.601892    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-tmp-dir\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:20:23.603721 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.603693    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6bffe02-745d-4ce1-a344-c51ad20684dd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f6bffe02-745d-4ce1-a344-c51ad20684dd" (UID: "f6bffe02-745d-4ce1-a344-c51ad20684dd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:20:23.603849 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.603732    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-dshm" (OuterVolumeSpecName: "dshm") pod "f6bffe02-745d-4ce1-a344-c51ad20684dd" (UID: "f6bffe02-745d-4ce1-a344-c51ad20684dd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:20:23.603910 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.603872    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6bffe02-745d-4ce1-a344-c51ad20684dd-kube-api-access-jsvwc" (OuterVolumeSpecName: "kube-api-access-jsvwc") pod "f6bffe02-745d-4ce1-a344-c51ad20684dd" (UID: "f6bffe02-745d-4ce1-a344-c51ad20684dd"). InnerVolumeSpecName "kube-api-access-jsvwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:20:23.703097 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.703056    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-dshm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:20:23.703097 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.703092    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsvwc\" (UniqueName: \"kubernetes.io/projected/f6bffe02-745d-4ce1-a344-c51ad20684dd-kube-api-access-jsvwc\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:20:23.703097 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.703104    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6bffe02-745d-4ce1-a344-c51ad20684dd-model-cache\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:20:23.703339 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.703114    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6bffe02-745d-4ce1-a344-c51ad20684dd-tls-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:20:23.855027 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.854986    2581 generic.go:356] "Generic (PLEG): container finished" podID="f6bffe02-745d-4ce1-a344-c51ad20684dd" containerID="563600f293b6c98ae22f3f1472659a7ffa2f82dad6f7614bc722c64fc543744f" exitCode=0
Sep 14 13:20:23.855192 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.855031    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"
Sep 14 13:20:23.855192 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.855040    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts" event={"ID":"f6bffe02-745d-4ce1-a344-c51ad20684dd","Type":"ContainerDied","Data":"563600f293b6c98ae22f3f1472659a7ffa2f82dad6f7614bc722c64fc543744f"}
Sep 14 13:20:23.855192 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.855089    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts" event={"ID":"f6bffe02-745d-4ce1-a344-c51ad20684dd","Type":"ContainerDied","Data":"2047ee6c7806f95d036ca6579c82f49ff733542f5e87b376a84336fb7d263427"}
Sep 14 13:20:23.855192 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.855106    2581 scope.go:117] "RemoveContainer" containerID="563600f293b6c98ae22f3f1472659a7ffa2f82dad6f7614bc722c64fc543744f"
Sep 14 13:20:23.865020 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.865001    2581 scope.go:117] "RemoveContainer" containerID="563600f293b6c98ae22f3f1472659a7ffa2f82dad6f7614bc722c64fc543744f"
Sep 14 13:20:23.865349 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:20:23.865328    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"563600f293b6c98ae22f3f1472659a7ffa2f82dad6f7614bc722c64fc543744f\": container with ID starting with 563600f293b6c98ae22f3f1472659a7ffa2f82dad6f7614bc722c64fc543744f not found: ID does not exist" containerID="563600f293b6c98ae22f3f1472659a7ffa2f82dad6f7614bc722c64fc543744f"
Sep 14 13:20:23.865449 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.865356    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"563600f293b6c98ae22f3f1472659a7ffa2f82dad6f7614bc722c64fc543744f"} err="failed to get container status \"563600f293b6c98ae22f3f1472659a7ffa2f82dad6f7614bc722c64fc543744f\": rpc error: code = NotFound desc = could not find container \"563600f293b6c98ae22f3f1472659a7ffa2f82dad6f7614bc722c64fc543744f\": container with ID starting with 563600f293b6c98ae22f3f1472659a7ffa2f82dad6f7614bc722c64fc543744f not found: ID does not exist"
Sep 14 13:20:23.877449 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.877420    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"]
Sep 14 13:20:23.880925 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:23.880903    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-4c0b93/noten-model-4c0b93-kserve-5bc76fff47-h48ts"]
Sep 14 13:20:25.247041 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:25.247009    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6bffe02-745d-4ce1-a344-c51ad20684dd" path="/var/lib/kubelet/pods/f6bffe02-745d-4ce1-a344-c51ad20684dd/volumes"
Sep 14 13:20:26.424847 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:26.424809    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:30.075131 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:30.075094    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:30.128590 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:30.128558    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:34.712224 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.712187    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"]
Sep 14 13:20:34.712695 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.712680    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bffe02-745d-4ce1-a344-c51ad20684dd" containerName="main"
Sep 14 13:20:34.712746 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.712697    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bffe02-745d-4ce1-a344-c51ad20684dd" containerName="main"
Sep 14 13:20:34.712806 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.712795    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6bffe02-745d-4ce1-a344-c51ad20684dd" containerName="main"
Sep 14 13:20:34.717324 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.717304    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.720446 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.720425    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b3d6c5\"/\"e2e-conflict-b-c95ce5f6-kserve-self-signed-certs\""
Sep 14 13:20:34.726262 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.726239    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"]
Sep 14 13:20:34.809320 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.809286    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:34.814326 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.814299    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-dshm\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.814472 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.814332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-tmp-dir\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.814472 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.814425    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06595848-9d9a-48db-a554-ce41d96a1b5c-tls-certs\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.814585 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.814479    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-model-cache\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.814585 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.814516    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8vzv\" (UniqueName: \"kubernetes.io/projected/06595848-9d9a-48db-a554-ce41d96a1b5c-kube-api-access-v8vzv\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.814653 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.814609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-home\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.915667 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.915634    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06595848-9d9a-48db-a554-ce41d96a1b5c-tls-certs\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.915888 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.915680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-model-cache\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.915888 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.915703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8vzv\" (UniqueName: \"kubernetes.io/projected/06595848-9d9a-48db-a554-ce41d96a1b5c-kube-api-access-v8vzv\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.915888 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.915772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-home\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.915888 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.915799    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-dshm\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.915888 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.915826    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-tmp-dir\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.916159 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.916137    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-home\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.916330 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.916303    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-tmp-dir\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.916461 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.916303    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-model-cache\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.918198 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.918166    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-dshm\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.918357 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.918340    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06595848-9d9a-48db-a554-ce41d96a1b5c-tls-certs\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:34.923686 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:34.923663    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8vzv\" (UniqueName: \"kubernetes.io/projected/06595848-9d9a-48db-a554-ce41d96a1b5c-kube-api-access-v8vzv\") pod \"e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:35.027867 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:35.027770    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:35.303744 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:35.303572    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"]
Sep 14 13:20:35.306297 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:20:35.306269    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06595848_9d9a_48db_a554_ce41d96a1b5c.slice/crio-02d7ed94316b851fe1c0d2cc9df3592acf42155029fb46ac9ed44bd5147cd01b WatchSource:0}: Error finding container 02d7ed94316b851fe1c0d2cc9df3592acf42155029fb46ac9ed44bd5147cd01b: Status 404 returned error can't find the container with id 02d7ed94316b851fe1c0d2cc9df3592acf42155029fb46ac9ed44bd5147cd01b
Sep 14 13:20:35.922341 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:35.922297    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n" event={"ID":"06595848-9d9a-48db-a554-ce41d96a1b5c","Type":"ContainerStarted","Data":"6798e0c3749c58dd92e086cdabab4dbd3f4e8ca1e175860f765faf70a6530289"}
Sep 14 13:20:35.922341 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:35.922348    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n" event={"ID":"06595848-9d9a-48db-a554-ce41d96a1b5c","Type":"ContainerStarted","Data":"02d7ed94316b851fe1c0d2cc9df3592acf42155029fb46ac9ed44bd5147cd01b"}
Sep 14 13:20:35.941435 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:35.941379    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n" podStartSLOduration=1.9413636520000002 podStartE2EDuration="1.941363652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:20:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:20:35.938747095 +0000 UTC m=+1329.230080911" watchObservedRunningTime="2026-09-14 13:20:35.941363652 +0000 UTC m=+1329.232697468"
Sep 14 13:20:43.069745 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:43.069703    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:20:45.028115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:45.028075    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:45.028593 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:45.028133    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:45.041031 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:45.040999    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:45.411704 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:45.411666    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:45.947782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:45.947733    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:45.992501 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:45.992472    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:20:50.009906 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.009817    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"]
Sep 14 13:20:50.013912 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.013887    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.016242 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.016217    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-bc851d\"/\"distinct-2-w0-bc851d-kserve-self-signed-certs\""
Sep 14 13:20:50.022017 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.021989    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"]
Sep 14 13:20:50.033711 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.033679    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:50.070017 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.069975    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-dshm\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.070017 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.070019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-tmp-dir\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.070279 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.070093    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-home\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.070279 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.070156    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f19605b1-bc2b-4a43-92a3-5a79e739ffab-tls-certs\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.070279 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.070197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-model-cache\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.070279 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.070219    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55rf8\" (UniqueName: \"kubernetes.io/projected/f19605b1-bc2b-4a43-92a3-5a79e739ffab-kube-api-access-55rf8\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.170882 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.170843    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-model-cache\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.170882 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.170881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55rf8\" (UniqueName: \"kubernetes.io/projected/f19605b1-bc2b-4a43-92a3-5a79e739ffab-kube-api-access-55rf8\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.171124 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.170973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-dshm\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.171124 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.171008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-tmp-dir\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.171124 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.171053    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-home\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.171124 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.171096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f19605b1-bc2b-4a43-92a3-5a79e739ffab-tls-certs\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.171352 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.171323    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-model-cache\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.171431 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.171413    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-tmp-dir\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.171517 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.171496    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-home\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.173383 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.173360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-dshm\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.173518 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.173500    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f19605b1-bc2b-4a43-92a3-5a79e739ffab-tls-certs\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.178472 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.178447    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55rf8\" (UniqueName: \"kubernetes.io/projected/f19605b1-bc2b-4a43-92a3-5a79e739ffab-kube-api-access-55rf8\") pod \"distinct-2-w0-bc851d-kserve-68448d9657-kqp95\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") " pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.325598 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.325486    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:20:50.608372 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:50.604789    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"]
Sep 14 13:20:50.613212 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:20:50.613176    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf19605b1_bc2b_4a43_92a3_5a79e739ffab.slice/crio-0c4cf58f657b7347ac44c1049d18959c8a3a834b583291ee446fe8945b3e3493 WatchSource:0}: Error finding container 0c4cf58f657b7347ac44c1049d18959c8a3a834b583291ee446fe8945b3e3493: Status 404 returned error can't find the container with id 0c4cf58f657b7347ac44c1049d18959c8a3a834b583291ee446fe8945b3e3493
Sep 14 13:20:51.009576 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:51.009470    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95" event={"ID":"f19605b1-bc2b-4a43-92a3-5a79e739ffab","Type":"ContainerStarted","Data":"04618123b2ac34a723c4b0ea158fd82529b81efc17e0bd8719b5651b729a05f3"}
Sep 14 13:20:51.009576 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:51.009511    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95" event={"ID":"f19605b1-bc2b-4a43-92a3-5a79e739ffab","Type":"ContainerStarted","Data":"0c4cf58f657b7347ac44c1049d18959c8a3a834b583291ee446fe8945b3e3493"}
Sep 14 13:20:51.028412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:20:51.028345    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95" podStartSLOduration=2.028323857 podStartE2EDuration="2.028323857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:20:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:20:51.025624546 +0000 UTC m=+1344.316958364" watchObservedRunningTime="2026-09-14 13:20:51.028323857 +0000 UTC m=+1344.319657677"
Sep 14 13:21:00.325845 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:00.325799    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:21:00.326283 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:00.325864    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:21:00.338621 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:00.338592    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:21:01.081250 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:01.081217    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:21:04.633001 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:04.632967    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:10.203881 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:10.203835    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:10.260694 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:10.260657    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:11.313103 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.313075    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"]
Sep 14 13:21:11.316697 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.316674    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.319414 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.319390    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-14f3af\"/\"kube-root-ca.crt\""
Sep 14 13:21:11.320541 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.320506    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-14f3af\"/\"openshift-service-ca.crt\""
Sep 14 13:21:11.320667 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.320566    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14f3af\"/\"default-dockercfg-pbl7q\""
Sep 14 13:21:11.320667 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.320635    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14f3af\"/\"auth-test-model-14f3af-kserve-self-signed-certs\""
Sep 14 13:21:11.327919 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.327896    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:11.328073 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.328052    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"]
Sep 14 13:21:11.384656 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.384618    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-model-cache\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.384867 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.384686    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-tmp-dir\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.384937 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.384868    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsmmq\" (UniqueName: \"kubernetes.io/projected/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-kube-api-access-xsmmq\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.384937 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.384923    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-dshm\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.385046 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.384995    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-tls-certs\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.385094 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.385055    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-home\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.486608 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.486573    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-model-cache\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.486784 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.486636    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-tmp-dir\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.486784 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.486715    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsmmq\" (UniqueName: \"kubernetes.io/projected/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-kube-api-access-xsmmq\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.486784 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.486735    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-dshm\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.486784 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.486774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-tls-certs\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.486983 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.486807    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-home\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.487065 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.487036    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-model-cache\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.487127 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.487063    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-tmp-dir\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.487127 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.487101    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-home\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.489051 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.489028    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-dshm\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.489443 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.489421    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-tls-certs\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.494367 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.494339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsmmq\" (UniqueName: \"kubernetes.io/projected/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-kube-api-access-xsmmq\") pod \"auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.628466 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.628426    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:11.903588 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:11.903553    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"]
Sep 14 13:21:11.905379 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:21:11.905344    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1e51dfb_f589_49c9_bc1b_90293f6d1b2d.slice/crio-47ab5a4e44c89625d8b332b3fa00285e77457a818d423469d1dd00ab6ee6b995 WatchSource:0}: Error finding container 47ab5a4e44c89625d8b332b3fa00285e77457a818d423469d1dd00ab6ee6b995: Status 404 returned error can't find the container with id 47ab5a4e44c89625d8b332b3fa00285e77457a818d423469d1dd00ab6ee6b995
Sep 14 13:21:12.131773 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:12.131736    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr" event={"ID":"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d","Type":"ContainerStarted","Data":"7309c8696c7ede2c6baca3c2047b890a0701d44f9dee5639712d4862c562a8e4"}
Sep 14 13:21:12.131773 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:12.131778    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr" event={"ID":"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d","Type":"ContainerStarted","Data":"47ab5a4e44c89625d8b332b3fa00285e77457a818d423469d1dd00ab6ee6b995"}
Sep 14 13:21:12.149286 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:12.149239    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr" podStartSLOduration=1.149225372 podStartE2EDuration="1.149225372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:21:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:21:12.148197773 +0000 UTC m=+1365.439531590" watchObservedRunningTime="2026-09-14 13:21:12.149225372 +0000 UTC m=+1365.440559188"
Sep 14 13:21:15.956188 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:15.956147    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:21:21.629257 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:21.629217    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:21.629257 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:21.629270    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:21.641734 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:21.641708    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:22.196789 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:22.196759    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:21:43.502920 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:43.502887    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:43.604196 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:43.604143    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:43.762798 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:43.762711    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:21:56.431141 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:56.431107    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"]
Sep 14 13:21:56.451998 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:56.451966    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:58.123922 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:21:58.123888    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:03.934470 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:03.934438    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:11.450084 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.450029    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n" podUID="06595848-9d9a-48db-a554-ce41d96a1b5c" containerName="main" containerID="cri-o://6798e0c3749c58dd92e086cdabab4dbd3f4e8ca1e175860f765faf70a6530289" gracePeriod=45
Sep 14 13:22:11.703601 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.703515    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:22:11.817383 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.817345    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-home\") pod \"06595848-9d9a-48db-a554-ce41d96a1b5c\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") "
Sep 14 13:22:11.817620 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.817401    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-model-cache\") pod \"06595848-9d9a-48db-a554-ce41d96a1b5c\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") "
Sep 14 13:22:11.817620 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.817429    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06595848-9d9a-48db-a554-ce41d96a1b5c-tls-certs\") pod \"06595848-9d9a-48db-a554-ce41d96a1b5c\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") "
Sep 14 13:22:11.817620 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.817483    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8vzv\" (UniqueName: \"kubernetes.io/projected/06595848-9d9a-48db-a554-ce41d96a1b5c-kube-api-access-v8vzv\") pod \"06595848-9d9a-48db-a554-ce41d96a1b5c\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") "
Sep 14 13:22:11.817620 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.817554    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-tmp-dir\") pod \"06595848-9d9a-48db-a554-ce41d96a1b5c\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") "
Sep 14 13:22:11.817846 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.817655    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-dshm\") pod \"06595848-9d9a-48db-a554-ce41d96a1b5c\" (UID: \"06595848-9d9a-48db-a554-ce41d96a1b5c\") "
Sep 14 13:22:11.817846 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.817713    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-home" (OuterVolumeSpecName: "home") pod "06595848-9d9a-48db-a554-ce41d96a1b5c" (UID: "06595848-9d9a-48db-a554-ce41d96a1b5c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:22:11.817846 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.817741    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-model-cache" (OuterVolumeSpecName: "model-cache") pod "06595848-9d9a-48db-a554-ce41d96a1b5c" (UID: "06595848-9d9a-48db-a554-ce41d96a1b5c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:22:11.817846 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.817805    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "06595848-9d9a-48db-a554-ce41d96a1b5c" (UID: "06595848-9d9a-48db-a554-ce41d96a1b5c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:22:11.818059 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.818040    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-tmp-dir\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:22:11.818123 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.818067    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-home\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:22:11.818123 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.818093    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-model-cache\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:22:11.819950 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.819924    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06595848-9d9a-48db-a554-ce41d96a1b5c-kube-api-access-v8vzv" (OuterVolumeSpecName: "kube-api-access-v8vzv") pod "06595848-9d9a-48db-a554-ce41d96a1b5c" (UID: "06595848-9d9a-48db-a554-ce41d96a1b5c"). InnerVolumeSpecName "kube-api-access-v8vzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:22:11.820059 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.819958    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-dshm" (OuterVolumeSpecName: "dshm") pod "06595848-9d9a-48db-a554-ce41d96a1b5c" (UID: "06595848-9d9a-48db-a554-ce41d96a1b5c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:22:11.820059 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.819968    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06595848-9d9a-48db-a554-ce41d96a1b5c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "06595848-9d9a-48db-a554-ce41d96a1b5c" (UID: "06595848-9d9a-48db-a554-ce41d96a1b5c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:22:11.919340 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.919304    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06595848-9d9a-48db-a554-ce41d96a1b5c-tls-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:22:11.919340 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.919334    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8vzv\" (UniqueName: \"kubernetes.io/projected/06595848-9d9a-48db-a554-ce41d96a1b5c-kube-api-access-v8vzv\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:22:11.919340 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:11.919346    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06595848-9d9a-48db-a554-ce41d96a1b5c-dshm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:22:12.452515 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:12.452475    2581 generic.go:356] "Generic (PLEG): container finished" podID="06595848-9d9a-48db-a554-ce41d96a1b5c" containerID="6798e0c3749c58dd92e086cdabab4dbd3f4e8ca1e175860f765faf70a6530289" exitCode=0
Sep 14 13:22:12.452515 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:12.452518    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"
Sep 14 13:22:12.453049 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:12.452523    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n" event={"ID":"06595848-9d9a-48db-a554-ce41d96a1b5c","Type":"ContainerDied","Data":"6798e0c3749c58dd92e086cdabab4dbd3f4e8ca1e175860f765faf70a6530289"}
Sep 14 13:22:12.453049 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:12.452587    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n" event={"ID":"06595848-9d9a-48db-a554-ce41d96a1b5c","Type":"ContainerDied","Data":"02d7ed94316b851fe1c0d2cc9df3592acf42155029fb46ac9ed44bd5147cd01b"}
Sep 14 13:22:12.453049 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:12.452607    2581 scope.go:117] "RemoveContainer" containerID="6798e0c3749c58dd92e086cdabab4dbd3f4e8ca1e175860f765faf70a6530289"
Sep 14 13:22:12.462603 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:12.462583    2581 scope.go:117] "RemoveContainer" containerID="6798e0c3749c58dd92e086cdabab4dbd3f4e8ca1e175860f765faf70a6530289"
Sep 14 13:22:12.462906 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:22:12.462889    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6798e0c3749c58dd92e086cdabab4dbd3f4e8ca1e175860f765faf70a6530289\": container with ID starting with 6798e0c3749c58dd92e086cdabab4dbd3f4e8ca1e175860f765faf70a6530289 not found: ID does not exist" containerID="6798e0c3749c58dd92e086cdabab4dbd3f4e8ca1e175860f765faf70a6530289"
Sep 14 13:22:12.462960 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:12.462916    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6798e0c3749c58dd92e086cdabab4dbd3f4e8ca1e175860f765faf70a6530289"} err="failed to get container status \"6798e0c3749c58dd92e086cdabab4dbd3f4e8ca1e175860f765faf70a6530289\": rpc error: code = NotFound desc = could not find container \"6798e0c3749c58dd92e086cdabab4dbd3f4e8ca1e175860f765faf70a6530289\": container with ID starting with 6798e0c3749c58dd92e086cdabab4dbd3f4e8ca1e175860f765faf70a6530289 not found: ID does not exist"
Sep 14 13:22:12.475134 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:12.475105    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"]
Sep 14 13:22:12.478988 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:12.478962    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/e2e-conflict-b-c95ce5f6-kserve-566c55b48f-p522n"]
Sep 14 13:22:13.247182 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:13.247151    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06595848-9d9a-48db-a554-ce41d96a1b5c" path="/var/lib/kubelet/pods/06595848-9d9a-48db-a554-ce41d96a1b5c/volumes"
Sep 14 13:22:17.602835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.602791    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"]
Sep 14 13:22:17.603482 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.603461    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06595848-9d9a-48db-a554-ce41d96a1b5c" containerName="main"
Sep 14 13:22:17.603482 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.603484    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="06595848-9d9a-48db-a554-ce41d96a1b5c" containerName="main"
Sep 14 13:22:17.603679 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.603606    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="06595848-9d9a-48db-a554-ce41d96a1b5c" containerName="main"
Sep 14 13:22:17.608239 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.608212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.612124 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.612087    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-eff5e8\"/\"default-dockercfg-hhsnb\""
Sep 14 13:22:17.612124 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.612117    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-eff5e8\"/\"kube-root-ca.crt\""
Sep 14 13:22:17.612328 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.612230    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-eff5e8\"/\"auth-test-model-eff5e8-kserve-self-signed-certs\""
Sep 14 13:22:17.612389 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.612345    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-eff5e8\"/\"openshift-service-ca.crt\""
Sep 14 13:22:17.619950 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.619922    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"]
Sep 14 13:22:17.650226 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.650195    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:17.677510 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.677461    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113015c3-fc40-4151-a86b-c57555eb75f5-tls-certs\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.677717 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.677522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-model-cache\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.677717 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.677608    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-home\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.677717 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.677656    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-tmp-dir\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.677829 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.677748    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-dshm\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.677829 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.677770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x45zp\" (UniqueName: \"kubernetes.io/projected/113015c3-fc40-4151-a86b-c57555eb75f5-kube-api-access-x45zp\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.778445 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.778393    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-model-cache\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.778445 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.778447    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-home\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.778723 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.778491    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-tmp-dir\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.778723 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.778605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-dshm\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.778723 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.778635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x45zp\" (UniqueName: \"kubernetes.io/projected/113015c3-fc40-4151-a86b-c57555eb75f5-kube-api-access-x45zp\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.778874 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.778740    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113015c3-fc40-4151-a86b-c57555eb75f5-tls-certs\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.778931 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.778913    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-model-cache\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.778982 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.778958    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-tmp-dir\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.778982 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.778974    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-home\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.781032 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.781003    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-dshm\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.781232 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.781216    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113015c3-fc40-4151-a86b-c57555eb75f5-tls-certs\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.787395 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.787370    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x45zp\" (UniqueName: \"kubernetes.io/projected/113015c3-fc40-4151-a86b-c57555eb75f5-kube-api-access-x45zp\") pod \"auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:17.921762 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:17.921663    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:18.196263 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:18.196227    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"]
Sep 14 13:22:18.198695 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:22:18.198665    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod113015c3_fc40_4151_a86b_c57555eb75f5.slice/crio-29337822a84d13ce3c360a78cf7d5693673b7f7d017d1f108e9b74779466f4b1 WatchSource:0}: Error finding container 29337822a84d13ce3c360a78cf7d5693673b7f7d017d1f108e9b74779466f4b1: Status 404 returned error can't find the container with id 29337822a84d13ce3c360a78cf7d5693673b7f7d017d1f108e9b74779466f4b1
Sep 14 13:22:18.487331 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:18.487227    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls" event={"ID":"113015c3-fc40-4151-a86b-c57555eb75f5","Type":"ContainerStarted","Data":"67d9bb4e2838949c581b7f6d8441525ca071a33775d255a5541e465865108d1d"}
Sep 14 13:22:18.487331 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:18.487279    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls" event={"ID":"113015c3-fc40-4151-a86b-c57555eb75f5","Type":"ContainerStarted","Data":"29337822a84d13ce3c360a78cf7d5693673b7f7d017d1f108e9b74779466f4b1"}
Sep 14 13:22:18.506022 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:18.505961    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls" podStartSLOduration=1.505942295 podStartE2EDuration="1.505942295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:22:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:22:18.504294029 +0000 UTC m=+1431.795627847" watchObservedRunningTime="2026-09-14 13:22:18.505942295 +0000 UTC m=+1431.797276168"
Sep 14 13:22:22.366338 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:22.366296    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:22:27.921894 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:27.921853    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:27.922325 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:27.921930    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:27.934602 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:27.934570    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:28.557512 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:28.557480    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:22:31.839432 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:31.839398    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:32.655662 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:32.655630    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:33.420272 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:33.420242    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:34.136460 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:34.136429    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:34.950442 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:34.950409    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:35.629210 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:35.629178    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:36.344797 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:36.344762    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:37.148075 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:37.148045    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:37.799071 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:37.799031    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:39.566830 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:39.566802    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:46.976742 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:46.976707    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"]
Sep 14 13:22:46.977139 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:46.976939    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm" podUID="2f7cd24f-017b-4572-9897-331a916d5b2d" containerName="maas-api" containerID="cri-o://84f194357ed64b4575350b427a9ea17c7d8531c5d414371e1e2ced1d8877a7e7" gracePeriod=30
Sep 14 13:22:47.038958 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.038926    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:47.091773 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.091740    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"]
Sep 14 13:22:47.092323 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.092002    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv" podUID="f7e1829e-3ae5-4f5c-9cf9-56642927b55d" containerName="payload-processing" containerID="cri-o://e6a0521741e4cd9979c16cd86cf1cf58dbb5c2a1345be093efb4ffaf5fbe09e9" gracePeriod=30
Sep 14 13:22:47.109733 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.109450    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"]
Sep 14 13:22:47.111015 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.109700    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq" podUID="bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b" containerName="payload-pre-processing" containerID="cri-o://936b00ca0c25b3870c9d4e484f544ef47b65b2c8f1775dbee12feb6bf09608b5" gracePeriod=30
Sep 14 13:22:47.157146 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.157114    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:47.252960 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.252929    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:22:47.273483 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.273457    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:47.395338 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.395219    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdd89\" (UniqueName: \"kubernetes.io/projected/2f7cd24f-017b-4572-9897-331a916d5b2d-kube-api-access-vdd89\") pod \"2f7cd24f-017b-4572-9897-331a916d5b2d\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") "
Sep 14 13:22:47.395338 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.395265    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-metrics-certs\") pod \"2f7cd24f-017b-4572-9897-331a916d5b2d\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") "
Sep 14 13:22:47.395338 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.395303    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f7cd24f-017b-4572-9897-331a916d5b2d-openshift-service-ca\") pod \"2f7cd24f-017b-4572-9897-331a916d5b2d\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") "
Sep 14 13:22:47.395832 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.395475    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-maas-api-tls\") pod \"2f7cd24f-017b-4572-9897-331a916d5b2d\" (UID: \"2f7cd24f-017b-4572-9897-331a916d5b2d\") "
Sep 14 13:22:47.396138 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.396111    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f7cd24f-017b-4572-9897-331a916d5b2d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f7cd24f-017b-4572-9897-331a916d5b2d" (UID: "2f7cd24f-017b-4572-9897-331a916d5b2d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:22:47.397823 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.397791    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2f7cd24f-017b-4572-9897-331a916d5b2d" (UID: "2f7cd24f-017b-4572-9897-331a916d5b2d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:22:47.397939 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.397848    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f7cd24f-017b-4572-9897-331a916d5b2d-kube-api-access-vdd89" (OuterVolumeSpecName: "kube-api-access-vdd89") pod "2f7cd24f-017b-4572-9897-331a916d5b2d" (UID: "2f7cd24f-017b-4572-9897-331a916d5b2d"). InnerVolumeSpecName "kube-api-access-vdd89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:22:47.398056 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.398034    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f7cd24f-017b-4572-9897-331a916d5b2d" (UID: "2f7cd24f-017b-4572-9897-331a916d5b2d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:22:47.405815 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.405780    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:47.408789 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.408762    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"
Sep 14 13:22:47.411796 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.411772    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"
Sep 14 13:22:47.496232 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.496140    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7e1829e-3ae5-4f5c-9cf9-56642927b55d-plugins-config-volume\") pod \"f7e1829e-3ae5-4f5c-9cf9-56642927b55d\" (UID: \"f7e1829e-3ae5-4f5c-9cf9-56642927b55d\") "
Sep 14 13:22:47.496232 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.496216    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7qg6\" (UniqueName: \"kubernetes.io/projected/f7e1829e-3ae5-4f5c-9cf9-56642927b55d-kube-api-access-w7qg6\") pod \"f7e1829e-3ae5-4f5c-9cf9-56642927b55d\" (UID: \"f7e1829e-3ae5-4f5c-9cf9-56642927b55d\") "
Sep 14 13:22:47.496456 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.496298    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xp6gw\" (UniqueName: \"kubernetes.io/projected/bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b-kube-api-access-xp6gw\") pod \"bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b\" (UID: \"bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b\") "
Sep 14 13:22:47.496456 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.496321    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b-plugins-config-volume\") pod \"bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b\" (UID: \"bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b\") "
Sep 14 13:22:47.496689 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.496665    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-maas-api-tls\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:22:47.496835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.496694    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdd89\" (UniqueName: \"kubernetes.io/projected/2f7cd24f-017b-4572-9897-331a916d5b2d-kube-api-access-vdd89\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:22:47.496835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.496715    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f7cd24f-017b-4572-9897-331a916d5b2d-metrics-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:22:47.496835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.496728    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f7cd24f-017b-4572-9897-331a916d5b2d-openshift-service-ca\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:22:47.496835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.496599    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7e1829e-3ae5-4f5c-9cf9-56642927b55d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7e1829e-3ae5-4f5c-9cf9-56642927b55d" (UID: "f7e1829e-3ae5-4f5c-9cf9-56642927b55d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:22:47.496835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.496777    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b" (UID: "bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:22:47.498500 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.498477    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7e1829e-3ae5-4f5c-9cf9-56642927b55d-kube-api-access-w7qg6" (OuterVolumeSpecName: "kube-api-access-w7qg6") pod "f7e1829e-3ae5-4f5c-9cf9-56642927b55d" (UID: "f7e1829e-3ae5-4f5c-9cf9-56642927b55d"). InnerVolumeSpecName "kube-api-access-w7qg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:22:47.498669 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.498644    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b-kube-api-access-xp6gw" (OuterVolumeSpecName: "kube-api-access-xp6gw") pod "bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b" (UID: "bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b"). InnerVolumeSpecName "kube-api-access-xp6gw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:22:47.597839 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.597797    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp6gw\" (UniqueName: \"kubernetes.io/projected/bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b-kube-api-access-xp6gw\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:22:47.597839 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.597831    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:22:47.597839 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.597841    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7e1829e-3ae5-4f5c-9cf9-56642927b55d-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:22:47.597839 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.597849    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7qg6\" (UniqueName: \"kubernetes.io/projected/f7e1829e-3ae5-4f5c-9cf9-56642927b55d-kube-api-access-w7qg6\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:22:47.630778 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.630742    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:47.647899 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.647860    2581 generic.go:356] "Generic (PLEG): container finished" podID="f7e1829e-3ae5-4f5c-9cf9-56642927b55d" containerID="e6a0521741e4cd9979c16cd86cf1cf58dbb5c2a1345be093efb4ffaf5fbe09e9" exitCode=0
Sep 14 13:22:47.648070 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.647913    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"
Sep 14 13:22:47.648070 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.647952    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv" event={"ID":"f7e1829e-3ae5-4f5c-9cf9-56642927b55d","Type":"ContainerDied","Data":"e6a0521741e4cd9979c16cd86cf1cf58dbb5c2a1345be093efb4ffaf5fbe09e9"}
Sep 14 13:22:47.648070 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.648020    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv" event={"ID":"f7e1829e-3ae5-4f5c-9cf9-56642927b55d","Type":"ContainerDied","Data":"09338b7fa2e55dfe9bbc0fbd4ec4de2c0060ce53f3a36a096d3a8c6d26a85ad6"}
Sep 14 13:22:47.648070 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.648049    2581 scope.go:117] "RemoveContainer" containerID="e6a0521741e4cd9979c16cd86cf1cf58dbb5c2a1345be093efb4ffaf5fbe09e9"
Sep 14 13:22:47.649617 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.649586    2581 generic.go:356] "Generic (PLEG): container finished" podID="bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b" containerID="936b00ca0c25b3870c9d4e484f544ef47b65b2c8f1775dbee12feb6bf09608b5" exitCode=0
Sep 14 13:22:47.649741 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.649631    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"
Sep 14 13:22:47.649741 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.649645    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq" event={"ID":"bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b","Type":"ContainerDied","Data":"936b00ca0c25b3870c9d4e484f544ef47b65b2c8f1775dbee12feb6bf09608b5"}
Sep 14 13:22:47.649741 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.649701    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq" event={"ID":"bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b","Type":"ContainerDied","Data":"fb69a60673f92a45c0b10a5b6dbf9fbdd14230d855f030c0244ee32783dec39c"}
Sep 14 13:22:47.651577 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.651551    2581 generic.go:356] "Generic (PLEG): container finished" podID="2f7cd24f-017b-4572-9897-331a916d5b2d" containerID="84f194357ed64b4575350b427a9ea17c7d8531c5d414371e1e2ced1d8877a7e7" exitCode=0
Sep 14 13:22:47.651706 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.651599    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"
Sep 14 13:22:47.651706 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.651624    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm" event={"ID":"2f7cd24f-017b-4572-9897-331a916d5b2d","Type":"ContainerDied","Data":"84f194357ed64b4575350b427a9ea17c7d8531c5d414371e1e2ced1d8877a7e7"}
Sep 14 13:22:47.651706 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.651666    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm" event={"ID":"2f7cd24f-017b-4572-9897-331a916d5b2d","Type":"ContainerDied","Data":"48fe0f799ed240b309314dd81867f8a62fc818943c14bc722f42b117cfb65016"}
Sep 14 13:22:47.663485 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.663455    2581 scope.go:117] "RemoveContainer" containerID="e6a0521741e4cd9979c16cd86cf1cf58dbb5c2a1345be093efb4ffaf5fbe09e9"
Sep 14 13:22:47.665141 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:22:47.664369    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6a0521741e4cd9979c16cd86cf1cf58dbb5c2a1345be093efb4ffaf5fbe09e9\": container with ID starting with e6a0521741e4cd9979c16cd86cf1cf58dbb5c2a1345be093efb4ffaf5fbe09e9 not found: ID does not exist" containerID="e6a0521741e4cd9979c16cd86cf1cf58dbb5c2a1345be093efb4ffaf5fbe09e9"
Sep 14 13:22:47.665141 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.664410    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6a0521741e4cd9979c16cd86cf1cf58dbb5c2a1345be093efb4ffaf5fbe09e9"} err="failed to get container status \"e6a0521741e4cd9979c16cd86cf1cf58dbb5c2a1345be093efb4ffaf5fbe09e9\": rpc error: code = NotFound desc = could not find container \"e6a0521741e4cd9979c16cd86cf1cf58dbb5c2a1345be093efb4ffaf5fbe09e9\": container with ID starting with e6a0521741e4cd9979c16cd86cf1cf58dbb5c2a1345be093efb4ffaf5fbe09e9 not found: ID does not exist"
Sep 14 13:22:47.665141 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.664437    2581 scope.go:117] "RemoveContainer" containerID="936b00ca0c25b3870c9d4e484f544ef47b65b2c8f1775dbee12feb6bf09608b5"
Sep 14 13:22:47.677481 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.677458    2581 scope.go:117] "RemoveContainer" containerID="936b00ca0c25b3870c9d4e484f544ef47b65b2c8f1775dbee12feb6bf09608b5"
Sep 14 13:22:47.677853 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:22:47.677832    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"936b00ca0c25b3870c9d4e484f544ef47b65b2c8f1775dbee12feb6bf09608b5\": container with ID starting with 936b00ca0c25b3870c9d4e484f544ef47b65b2c8f1775dbee12feb6bf09608b5 not found: ID does not exist" containerID="936b00ca0c25b3870c9d4e484f544ef47b65b2c8f1775dbee12feb6bf09608b5"
Sep 14 13:22:47.677941 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.677863    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"936b00ca0c25b3870c9d4e484f544ef47b65b2c8f1775dbee12feb6bf09608b5"} err="failed to get container status \"936b00ca0c25b3870c9d4e484f544ef47b65b2c8f1775dbee12feb6bf09608b5\": rpc error: code = NotFound desc = could not find container \"936b00ca0c25b3870c9d4e484f544ef47b65b2c8f1775dbee12feb6bf09608b5\": container with ID starting with 936b00ca0c25b3870c9d4e484f544ef47b65b2c8f1775dbee12feb6bf09608b5 not found: ID does not exist"
Sep 14 13:22:47.677941 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.677882    2581 scope.go:117] "RemoveContainer" containerID="84f194357ed64b4575350b427a9ea17c7d8531c5d414371e1e2ced1d8877a7e7"
Sep 14 13:22:47.687841 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.687813    2581 scope.go:117] "RemoveContainer" containerID="84f194357ed64b4575350b427a9ea17c7d8531c5d414371e1e2ced1d8877a7e7"
Sep 14 13:22:47.688220 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:22:47.688199    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84f194357ed64b4575350b427a9ea17c7d8531c5d414371e1e2ced1d8877a7e7\": container with ID starting with 84f194357ed64b4575350b427a9ea17c7d8531c5d414371e1e2ced1d8877a7e7 not found: ID does not exist" containerID="84f194357ed64b4575350b427a9ea17c7d8531c5d414371e1e2ced1d8877a7e7"
Sep 14 13:22:47.688285 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.688237    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84f194357ed64b4575350b427a9ea17c7d8531c5d414371e1e2ced1d8877a7e7"} err="failed to get container status \"84f194357ed64b4575350b427a9ea17c7d8531c5d414371e1e2ced1d8877a7e7\": rpc error: code = NotFound desc = could not find container \"84f194357ed64b4575350b427a9ea17c7d8531c5d414371e1e2ced1d8877a7e7\": container with ID starting with 84f194357ed64b4575350b427a9ea17c7d8531c5d414371e1e2ced1d8877a7e7 not found: ID does not exist"
Sep 14 13:22:47.693937 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.693905    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"]
Sep 14 13:22:47.699460 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.699430    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-336ccc18-6f675b7498-fjpkq"]
Sep 14 13:22:47.717221 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.717153    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"]
Sep 14 13:22:47.721122 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.721093    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-336ccc18-7f494f8ffc-x4bhv"]
Sep 14 13:22:47.735525 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.735495    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"]
Sep 14 13:22:47.739934 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:47.739905    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-336ccc18-86686c6bb-lw7hm"]
Sep 14 13:22:48.514588 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:48.514552    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:49.246569 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:49.246516    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f7cd24f-017b-4572-9897-331a916d5b2d" path="/var/lib/kubelet/pods/2f7cd24f-017b-4572-9897-331a916d5b2d/volumes"
Sep 14 13:22:49.246943 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:49.246928    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b" path="/var/lib/kubelet/pods/bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b/volumes"
Sep 14 13:22:49.247255 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:49.247244    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7e1829e-3ae5-4f5c-9cf9-56642927b55d" path="/var/lib/kubelet/pods/f7e1829e-3ae5-4f5c-9cf9-56642927b55d/volumes"
Sep 14 13:22:49.590824 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:49.590738    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:50.462517 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:50.462469    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:50.846558 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:50.846441    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:51.341039 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:51.341002    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:52.378497 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:52.378459    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:53.199033 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:53.198981    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:53.918700 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:53.918648    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:54.778895 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:54.778848    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:55.890662 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:55.890630    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:57.569773 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:22:57.569737    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:23:01.755352 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.755309    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"]
Sep 14 13:23:01.755813 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.755797    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f7cd24f-017b-4572-9897-331a916d5b2d" containerName="maas-api"
Sep 14 13:23:01.755905 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.755815    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f7cd24f-017b-4572-9897-331a916d5b2d" containerName="maas-api"
Sep 14 13:23:01.755905 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.755835    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e1829e-3ae5-4f5c-9cf9-56642927b55d" containerName="payload-processing"
Sep 14 13:23:01.755905 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.755841    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e1829e-3ae5-4f5c-9cf9-56642927b55d" containerName="payload-processing"
Sep 14 13:23:01.755905 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.755859    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b" containerName="payload-pre-processing"
Sep 14 13:23:01.755905 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.755865    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b" containerName="payload-pre-processing"
Sep 14 13:23:01.756059 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.755927    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7e1829e-3ae5-4f5c-9cf9-56642927b55d" containerName="payload-processing"
Sep 14 13:23:01.756059 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.755936    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcfbd3eb-e27b-466b-903f-8f22ef6b1c8b" containerName="payload-pre-processing"
Sep 14 13:23:01.756059 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.755943    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f7cd24f-017b-4572-9897-331a916d5b2d" containerName="maas-api"
Sep 14 13:23:01.758669 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.758645    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:01.762840 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.762816    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b3d6c5\"/\"distinct-2-w4-b3d6c5-kserve-self-signed-certs\""
Sep 14 13:23:01.777615 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.777586    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"]
Sep 14 13:23:01.952931 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.952897    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-tmp-dir\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:01.953113 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.952937    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-tls-certs\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:01.953113 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.953048    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrhnz\" (UniqueName: \"kubernetes.io/projected/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-kube-api-access-lrhnz\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:01.953187 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.953112    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-model-cache\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:01.953187 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.953155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-home\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:01.953253 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:01.953228    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-dshm\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:02.054860 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.054768    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-model-cache\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:02.055035 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.054936    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-home\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:02.055105 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.055035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-dshm\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:02.055105 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.055068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-tmp-dir\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:02.055226 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.055103    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-tls-certs\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:02.055226 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.055197    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrhnz\" (UniqueName: \"kubernetes.io/projected/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-kube-api-access-lrhnz\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:02.055332 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.055233    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-model-cache\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:02.055332 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.055318    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-home\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:02.055413 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.055375    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-tmp-dir\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:02.057305 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.057267    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-dshm\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:02.057629 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.057612    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-tls-certs\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:02.067506 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.067484    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrhnz\" (UniqueName: \"kubernetes.io/projected/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-kube-api-access-lrhnz\") pod \"distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") " pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:02.068348 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.068328    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:02.070386 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.070363    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:02.380341 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.380309    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"]
Sep 14 13:23:02.383661 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:23:02.383633    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cf9609a_f243_49b7_8847_3e7aaafb1d3c.slice/crio-9df0086b4648f9dfa19d16be547647969a668f8e013f7dff726b74166676efc4 WatchSource:0}: Error finding container 9df0086b4648f9dfa19d16be547647969a668f8e013f7dff726b74166676efc4: Status 404 returned error can't find the container with id 9df0086b4648f9dfa19d16be547647969a668f8e013f7dff726b74166676efc4
Sep 14 13:23:02.742174 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.742055    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z" event={"ID":"4cf9609a-f243-49b7-8847-3e7aaafb1d3c","Type":"ContainerStarted","Data":"443093c4f92e03baaef838d0c7c30d31fcc081ecee8c712d289df7b057527c70"}
Sep 14 13:23:02.742174 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.742115    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z" event={"ID":"4cf9609a-f243-49b7-8847-3e7aaafb1d3c","Type":"ContainerStarted","Data":"9df0086b4648f9dfa19d16be547647969a668f8e013f7dff726b74166676efc4"}
Sep 14 13:23:02.775545 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:02.775477    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z" podStartSLOduration=1.7754593650000001 podStartE2EDuration="1.775459365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:23:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:23:02.773639607 +0000 UTC m=+1476.064973424" watchObservedRunningTime="2026-09-14 13:23:02.775459365 +0000 UTC m=+1476.066793174"
Sep 14 13:23:11.366625 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:11.366592    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:12.068982 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:12.068951    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:12.069195 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:12.068998    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:12.082069 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:12.082041    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:12.805795 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:12.805764    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:23:27.253435 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:27.253402    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:23:27.255613 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:27.255589    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:23:30.325550 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:30.325495    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:30.685121 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:30.685086    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:23:31.392157 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:31.392115    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:31.466472 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:31.466440    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:33.812524 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:33.812490    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:33.913377 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:33.913343    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:40.333026 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:40.332985    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:49.406668 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:23:49.406581    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:14.560115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:14.560079    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:24:22.383220 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:22.383181    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:23.842719 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:23.842684    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:24.575561 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:24.575501    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:24.677469 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:24.677441    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:27.015813 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:27.015778    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:27.076794 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:27.076764    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:30.340496 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:30.340458    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:38.271251 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:38.271218    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:38.631216 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:38.631183    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:38.917462 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:38.917375    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:39.382903 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:39.382863    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:40.029035 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:40.029001    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:40.751139 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:40.751106    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:41.403674 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:41.403639    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:42.059286 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:42.059238    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:42.850075 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:42.850016    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:43.487104 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:43.487047    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:45.016843 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:45.016808    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:51.456650 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:51.456609    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:24:57.846185 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:24:57.846149    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:25:27.369540 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:25:27.369504    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:25:55.301226 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:25:55.301173    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:25:59.753691 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:25:59.753651    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:26:05.867933 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:05.867896    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:06.210914 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:06.210828    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:06.938323 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:06.938285    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:07.748263 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:07.748233    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:08.469754 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:08.469719    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:09.404089 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:09.404057    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:09.910571 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:09.910513    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:10.537419 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:10.537384    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:11.365245 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:11.365166    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:12.735575 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:12.735512    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:23.293388 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:23.293353    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:26.449666 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:26.449622    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"]
Sep 14 13:26:26.460407 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:26.460377    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"]
Sep 14 13:26:26.479623 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:26.479588    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"]
Sep 14 13:26:26.481999 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:26.481971    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"]
Sep 14 13:26:26.523850 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:26.523816    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"]
Sep 14 13:26:26.580205 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:26.580168    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:26.642172 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:26.642139    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:28.748410 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:28.748371    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:29.739247 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:29.739212    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:29.849503 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:29.849469    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:29.939817 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:29.939785    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:30.385762 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:30.385728    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:30.483067 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:30.483033    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:34.254159 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:34.254116    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:26:34.341750 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:34.341713    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:34.413816 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:34.413782    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:34.515680 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:34.515588    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:34.588880 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:34.588844    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:34.907127 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:34.907092    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:35.005604 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:35.005574    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:35.621897 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:35.621865    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:36.351204 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:36.351174    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:37.070705 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:37.070669    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:37.729627 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:37.729597    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:38.437817 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:38.437782    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:40.397027 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:40.396995    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:40.643777 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:40.643747    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:40.690224 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:40.690140    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"]
Sep 14 13:26:40.722665 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:40.722629    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:41.476487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.476445    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95" podUID="f19605b1-bc2b-4a43-92a3-5a79e739ffab" containerName="main" containerID="cri-o://04618123b2ac34a723c4b0ea158fd82529b81efc17e0bd8719b5651b729a05f3" gracePeriod=45
Sep 14 13:26:41.481791 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.481759    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x" podUID="5c7d4e50-3882-439d-8202-bb7efdb6e8e9" containerName="main" containerID="cri-o://1e03508d655a6539ade48144730ebe6e8b7ac1662f08c1413aac8a8b37697261" gracePeriod=45
Sep 14 13:26:41.502025 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.501989    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw" podUID="ef6e3627-df7d-404f-9e0c-a069dbc5d8f7" containerName="main" containerID="cri-o://8949a8cf0d3ce63cfb62d6f04f7a1573d10889bec869e2816cdbd43b4da23219" gracePeriod=45
Sep 14 13:26:41.506952 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.506921    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z" podUID="4cf9609a-f243-49b7-8847-3e7aaafb1d3c" containerName="main" containerID="cri-o://443093c4f92e03baaef838d0c7c30d31fcc081ecee8c712d289df7b057527c70" gracePeriod=45
Sep 14 13:26:41.547038 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.546997    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4" podUID="8b71224c-5355-4319-a590-56105f625d2f" containerName="main" containerID="cri-o://8303058cab88631662c57bb0967f709e01d16fc10b3fe7b981ae88974956038c" gracePeriod=45
Sep 14 13:26:41.849012 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.848968    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:26:41.894843 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.894726    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:26:41.908947 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.908919    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:26:41.911183 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.911157    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:26:41.913355 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.913312    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:26:41.929682 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.929180    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-tmp-dir\") pod \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") "
Sep 14 13:26:41.929682 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.929230    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55rf8\" (UniqueName: \"kubernetes.io/projected/f19605b1-bc2b-4a43-92a3-5a79e739ffab-kube-api-access-55rf8\") pod \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") "
Sep 14 13:26:41.929682 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.929269    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-model-cache\") pod \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") "
Sep 14 13:26:41.929682 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.929307    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f19605b1-bc2b-4a43-92a3-5a79e739ffab-tls-certs\") pod \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") "
Sep 14 13:26:41.929682 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.929397    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-dshm\") pod \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") "
Sep 14 13:26:41.929682 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.929439    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-home\") pod \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\" (UID: \"f19605b1-bc2b-4a43-92a3-5a79e739ffab\") "
Sep 14 13:26:41.930154 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.929948    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f19605b1-bc2b-4a43-92a3-5a79e739ffab" (UID: "f19605b1-bc2b-4a43-92a3-5a79e739ffab"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:41.930154 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.929957    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-home" (OuterVolumeSpecName: "home") pod "f19605b1-bc2b-4a43-92a3-5a79e739ffab" (UID: "f19605b1-bc2b-4a43-92a3-5a79e739ffab"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:41.930154 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.930139    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-model-cache" (OuterVolumeSpecName: "model-cache") pod "f19605b1-bc2b-4a43-92a3-5a79e739ffab" (UID: "f19605b1-bc2b-4a43-92a3-5a79e739ffab"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:41.932052 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.932027    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f19605b1-bc2b-4a43-92a3-5a79e739ffab-kube-api-access-55rf8" (OuterVolumeSpecName: "kube-api-access-55rf8") pod "f19605b1-bc2b-4a43-92a3-5a79e739ffab" (UID: "f19605b1-bc2b-4a43-92a3-5a79e739ffab"). InnerVolumeSpecName "kube-api-access-55rf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:26:41.932510 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.932485    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-dshm" (OuterVolumeSpecName: "dshm") pod "f19605b1-bc2b-4a43-92a3-5a79e739ffab" (UID: "f19605b1-bc2b-4a43-92a3-5a79e739ffab"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:41.932890 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.932865    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f19605b1-bc2b-4a43-92a3-5a79e739ffab-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f19605b1-bc2b-4a43-92a3-5a79e739ffab" (UID: "f19605b1-bc2b-4a43-92a3-5a79e739ffab"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:26:41.934711 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.934682    2581 generic.go:356] "Generic (PLEG): container finished" podID="ef6e3627-df7d-404f-9e0c-a069dbc5d8f7" containerID="8949a8cf0d3ce63cfb62d6f04f7a1573d10889bec869e2816cdbd43b4da23219" exitCode=0
Sep 14 13:26:41.934849 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.934723    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"
Sep 14 13:26:41.934849 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.934776    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw" event={"ID":"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7","Type":"ContainerDied","Data":"8949a8cf0d3ce63cfb62d6f04f7a1573d10889bec869e2816cdbd43b4da23219"}
Sep 14 13:26:41.934849 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.934823    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw" event={"ID":"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7","Type":"ContainerDied","Data":"db6f9fd4f41262a35373f82606ad4b0acd756b888194016a78c7e700804ed6ad"}
Sep 14 13:26:41.934849 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.934845    2581 scope.go:117] "RemoveContainer" containerID="8949a8cf0d3ce63cfb62d6f04f7a1573d10889bec869e2816cdbd43b4da23219"
Sep 14 13:26:41.936896 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.936870    2581 generic.go:356] "Generic (PLEG): container finished" podID="f19605b1-bc2b-4a43-92a3-5a79e739ffab" containerID="04618123b2ac34a723c4b0ea158fd82529b81efc17e0bd8719b5651b729a05f3" exitCode=0
Sep 14 13:26:41.937022 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.936977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95" event={"ID":"f19605b1-bc2b-4a43-92a3-5a79e739ffab","Type":"ContainerDied","Data":"04618123b2ac34a723c4b0ea158fd82529b81efc17e0bd8719b5651b729a05f3"}
Sep 14 13:26:41.937022 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.937014    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95" event={"ID":"f19605b1-bc2b-4a43-92a3-5a79e739ffab","Type":"ContainerDied","Data":"0c4cf58f657b7347ac44c1049d18959c8a3a834b583291ee446fe8945b3e3493"}
Sep 14 13:26:41.937143 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.937076    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"
Sep 14 13:26:41.938728 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.938705    2581 generic.go:356] "Generic (PLEG): container finished" podID="8b71224c-5355-4319-a590-56105f625d2f" containerID="8303058cab88631662c57bb0967f709e01d16fc10b3fe7b981ae88974956038c" exitCode=0
Sep 14 13:26:41.938853 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.938757    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"
Sep 14 13:26:41.938853 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.938793    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4" event={"ID":"8b71224c-5355-4319-a590-56105f625d2f","Type":"ContainerDied","Data":"8303058cab88631662c57bb0967f709e01d16fc10b3fe7b981ae88974956038c"}
Sep 14 13:26:41.938853 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.938829    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4" event={"ID":"8b71224c-5355-4319-a590-56105f625d2f","Type":"ContainerDied","Data":"35b723b1a81500895b638edda80c45fc98fcabc60b86b2501562f54d456a1e9d"}
Sep 14 13:26:41.940482 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.940458    2581 generic.go:356] "Generic (PLEG): container finished" podID="4cf9609a-f243-49b7-8847-3e7aaafb1d3c" containerID="443093c4f92e03baaef838d0c7c30d31fcc081ecee8c712d289df7b057527c70" exitCode=0
Sep 14 13:26:41.940616 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.940489    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z" event={"ID":"4cf9609a-f243-49b7-8847-3e7aaafb1d3c","Type":"ContainerDied","Data":"443093c4f92e03baaef838d0c7c30d31fcc081ecee8c712d289df7b057527c70"}
Sep 14 13:26:41.940616 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.940505    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"
Sep 14 13:26:41.940616 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.940524    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z" event={"ID":"4cf9609a-f243-49b7-8847-3e7aaafb1d3c","Type":"ContainerDied","Data":"9df0086b4648f9dfa19d16be547647969a668f8e013f7dff726b74166676efc4"}
Sep 14 13:26:41.945250 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.945222    2581 generic.go:356] "Generic (PLEG): container finished" podID="5c7d4e50-3882-439d-8202-bb7efdb6e8e9" containerID="1e03508d655a6539ade48144730ebe6e8b7ac1662f08c1413aac8a8b37697261" exitCode=0
Sep 14 13:26:41.945368 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.945264    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x" event={"ID":"5c7d4e50-3882-439d-8202-bb7efdb6e8e9","Type":"ContainerDied","Data":"1e03508d655a6539ade48144730ebe6e8b7ac1662f08c1413aac8a8b37697261"}
Sep 14 13:26:41.945368 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.945271    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"
Sep 14 13:26:41.945368 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.945293    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x" event={"ID":"5c7d4e50-3882-439d-8202-bb7efdb6e8e9","Type":"ContainerDied","Data":"4a328a1f554157cef791b3d475de1f5ed4142c5bcef4acd5ec1a76a231f33ba7"}
Sep 14 13:26:41.948259 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.948237    2581 scope.go:117] "RemoveContainer" containerID="8949a8cf0d3ce63cfb62d6f04f7a1573d10889bec869e2816cdbd43b4da23219"
Sep 14 13:26:41.948768 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:26:41.948741    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8949a8cf0d3ce63cfb62d6f04f7a1573d10889bec869e2816cdbd43b4da23219\": container with ID starting with 8949a8cf0d3ce63cfb62d6f04f7a1573d10889bec869e2816cdbd43b4da23219 not found: ID does not exist" containerID="8949a8cf0d3ce63cfb62d6f04f7a1573d10889bec869e2816cdbd43b4da23219"
Sep 14 13:26:41.948876 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.948780    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8949a8cf0d3ce63cfb62d6f04f7a1573d10889bec869e2816cdbd43b4da23219"} err="failed to get container status \"8949a8cf0d3ce63cfb62d6f04f7a1573d10889bec869e2816cdbd43b4da23219\": rpc error: code = NotFound desc = could not find container \"8949a8cf0d3ce63cfb62d6f04f7a1573d10889bec869e2816cdbd43b4da23219\": container with ID starting with 8949a8cf0d3ce63cfb62d6f04f7a1573d10889bec869e2816cdbd43b4da23219 not found: ID does not exist"
Sep 14 13:26:41.948876 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.948822    2581 scope.go:117] "RemoveContainer" containerID="04618123b2ac34a723c4b0ea158fd82529b81efc17e0bd8719b5651b729a05f3"
Sep 14 13:26:41.960164 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.960139    2581 scope.go:117] "RemoveContainer" containerID="04618123b2ac34a723c4b0ea158fd82529b81efc17e0bd8719b5651b729a05f3"
Sep 14 13:26:41.960574 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:26:41.960518    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04618123b2ac34a723c4b0ea158fd82529b81efc17e0bd8719b5651b729a05f3\": container with ID starting with 04618123b2ac34a723c4b0ea158fd82529b81efc17e0bd8719b5651b729a05f3 not found: ID does not exist" containerID="04618123b2ac34a723c4b0ea158fd82529b81efc17e0bd8719b5651b729a05f3"
Sep 14 13:26:41.960684 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.960583    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04618123b2ac34a723c4b0ea158fd82529b81efc17e0bd8719b5651b729a05f3"} err="failed to get container status \"04618123b2ac34a723c4b0ea158fd82529b81efc17e0bd8719b5651b729a05f3\": rpc error: code = NotFound desc = could not find container \"04618123b2ac34a723c4b0ea158fd82529b81efc17e0bd8719b5651b729a05f3\": container with ID starting with 04618123b2ac34a723c4b0ea158fd82529b81efc17e0bd8719b5651b729a05f3 not found: ID does not exist"
Sep 14 13:26:41.960684 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.960613    2581 scope.go:117] "RemoveContainer" containerID="8303058cab88631662c57bb0967f709e01d16fc10b3fe7b981ae88974956038c"
Sep 14 13:26:41.972391 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.972365    2581 scope.go:117] "RemoveContainer" containerID="8303058cab88631662c57bb0967f709e01d16fc10b3fe7b981ae88974956038c"
Sep 14 13:26:41.972840 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:26:41.972813    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8303058cab88631662c57bb0967f709e01d16fc10b3fe7b981ae88974956038c\": container with ID starting with 8303058cab88631662c57bb0967f709e01d16fc10b3fe7b981ae88974956038c not found: ID does not exist" containerID="8303058cab88631662c57bb0967f709e01d16fc10b3fe7b981ae88974956038c"
Sep 14 13:26:41.972947 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.972851    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8303058cab88631662c57bb0967f709e01d16fc10b3fe7b981ae88974956038c"} err="failed to get container status \"8303058cab88631662c57bb0967f709e01d16fc10b3fe7b981ae88974956038c\": rpc error: code = NotFound desc = could not find container \"8303058cab88631662c57bb0967f709e01d16fc10b3fe7b981ae88974956038c\": container with ID starting with 8303058cab88631662c57bb0967f709e01d16fc10b3fe7b981ae88974956038c not found: ID does not exist"
Sep 14 13:26:41.972947 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.972871    2581 scope.go:117] "RemoveContainer" containerID="443093c4f92e03baaef838d0c7c30d31fcc081ecee8c712d289df7b057527c70"
Sep 14 13:26:41.983797 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.983724    2581 scope.go:117] "RemoveContainer" containerID="443093c4f92e03baaef838d0c7c30d31fcc081ecee8c712d289df7b057527c70"
Sep 14 13:26:41.984307 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:26:41.984227    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"443093c4f92e03baaef838d0c7c30d31fcc081ecee8c712d289df7b057527c70\": container with ID starting with 443093c4f92e03baaef838d0c7c30d31fcc081ecee8c712d289df7b057527c70 not found: ID does not exist" containerID="443093c4f92e03baaef838d0c7c30d31fcc081ecee8c712d289df7b057527c70"
Sep 14 13:26:41.984627 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.984303    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"443093c4f92e03baaef838d0c7c30d31fcc081ecee8c712d289df7b057527c70"} err="failed to get container status \"443093c4f92e03baaef838d0c7c30d31fcc081ecee8c712d289df7b057527c70\": rpc error: code = NotFound desc = could not find container \"443093c4f92e03baaef838d0c7c30d31fcc081ecee8c712d289df7b057527c70\": container with ID starting with 443093c4f92e03baaef838d0c7c30d31fcc081ecee8c712d289df7b057527c70 not found: ID does not exist"
Sep 14 13:26:41.984627 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.984339    2581 scope.go:117] "RemoveContainer" containerID="1e03508d655a6539ade48144730ebe6e8b7ac1662f08c1413aac8a8b37697261"
Sep 14 13:26:41.996414 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.996392    2581 scope.go:117] "RemoveContainer" containerID="1e03508d655a6539ade48144730ebe6e8b7ac1662f08c1413aac8a8b37697261"
Sep 14 13:26:41.996805 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:26:41.996782    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e03508d655a6539ade48144730ebe6e8b7ac1662f08c1413aac8a8b37697261\": container with ID starting with 1e03508d655a6539ade48144730ebe6e8b7ac1662f08c1413aac8a8b37697261 not found: ID does not exist" containerID="1e03508d655a6539ade48144730ebe6e8b7ac1662f08c1413aac8a8b37697261"
Sep 14 13:26:41.996899 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:41.996815    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e03508d655a6539ade48144730ebe6e8b7ac1662f08c1413aac8a8b37697261"} err="failed to get container status \"1e03508d655a6539ade48144730ebe6e8b7ac1662f08c1413aac8a8b37697261\": rpc error: code = NotFound desc = could not find container \"1e03508d655a6539ade48144730ebe6e8b7ac1662f08c1413aac8a8b37697261\": container with ID starting with 1e03508d655a6539ade48144730ebe6e8b7ac1662f08c1413aac8a8b37697261 not found: ID does not exist"
Sep 14 13:26:42.015449 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.015412    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"]
Sep 14 13:26:42.017585 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.017558    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/distinct-2-w0-bc851d-kserve-68448d9657-kqp95"]
Sep 14 13:26:42.031006 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.030925    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-tmp-dir\") pod \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") "
Sep 14 13:26:42.031006 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.030978    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b71224c-5355-4319-a590-56105f625d2f-tls-certs\") pod \"8b71224c-5355-4319-a590-56105f625d2f\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") "
Sep 14 13:26:42.031212 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031012    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-tls-certs\") pod \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") "
Sep 14 13:26:42.031212 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031046    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwgmn\" (UniqueName: \"kubernetes.io/projected/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-kube-api-access-pwgmn\") pod \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") "
Sep 14 13:26:42.031212 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031062    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-home\") pod \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") "
Sep 14 13:26:42.031212 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031094    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-home\") pod \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") "
Sep 14 13:26:42.031430 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031258    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ef6e3627-df7d-404f-9e0c-a069dbc5d8f7" (UID: "ef6e3627-df7d-404f-9e0c-a069dbc5d8f7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.031430 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031309    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-model-cache\") pod \"8b71224c-5355-4319-a590-56105f625d2f\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") "
Sep 14 13:26:42.031430 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031343    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-home" (OuterVolumeSpecName: "home") pod "4cf9609a-f243-49b7-8847-3e7aaafb1d3c" (UID: "4cf9609a-f243-49b7-8847-3e7aaafb1d3c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.031430 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031360    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-home\") pod \"8b71224c-5355-4319-a590-56105f625d2f\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") "
Sep 14 13:26:42.031430 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031361    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-home" (OuterVolumeSpecName: "home") pod "ef6e3627-df7d-404f-9e0c-a069dbc5d8f7" (UID: "ef6e3627-df7d-404f-9e0c-a069dbc5d8f7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.031430 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031399    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrhnz\" (UniqueName: \"kubernetes.io/projected/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-kube-api-access-lrhnz\") pod \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") "
Sep 14 13:26:42.031770 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031494    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-model-cache\") pod \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") "
Sep 14 13:26:42.031770 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031566    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-model-cache" (OuterVolumeSpecName: "model-cache") pod "8b71224c-5355-4319-a590-56105f625d2f" (UID: "8b71224c-5355-4319-a590-56105f625d2f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.031770 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031581    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-home" (OuterVolumeSpecName: "home") pod "8b71224c-5355-4319-a590-56105f625d2f" (UID: "8b71224c-5355-4319-a590-56105f625d2f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.031770 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031729    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-model-cache" (OuterVolumeSpecName: "model-cache") pod "5c7d4e50-3882-439d-8202-bb7efdb6e8e9" (UID: "5c7d4e50-3882-439d-8202-bb7efdb6e8e9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.031982 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031774    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-tls-certs\") pod \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") "
Sep 14 13:26:42.031982 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031831    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7d22\" (UniqueName: \"kubernetes.io/projected/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-kube-api-access-l7d22\") pod \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") "
Sep 14 13:26:42.031982 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031866    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-tls-certs\") pod \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") "
Sep 14 13:26:42.031982 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031899    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-model-cache\") pod \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") "
Sep 14 13:26:42.031982 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031928    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-tmp-dir\") pod \"8b71224c-5355-4319-a590-56105f625d2f\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") "
Sep 14 13:26:42.032245 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.031987    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-model-cache\") pod \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") "
Sep 14 13:26:42.032245 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032023    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-tmp-dir\") pod \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") "
Sep 14 13:26:42.032245 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032058    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-dshm\") pod \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\" (UID: \"ef6e3627-df7d-404f-9e0c-a069dbc5d8f7\") "
Sep 14 13:26:42.032245 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032082    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hnzv\" (UniqueName: \"kubernetes.io/projected/8b71224c-5355-4319-a590-56105f625d2f-kube-api-access-6hnzv\") pod \"8b71224c-5355-4319-a590-56105f625d2f\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") "
Sep 14 13:26:42.032245 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032145    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-dshm\") pod \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") "
Sep 14 13:26:42.032245 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032177    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-tmp-dir\") pod \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\" (UID: \"4cf9609a-f243-49b7-8847-3e7aaafb1d3c\") "
Sep 14 13:26:42.032245 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032202    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-dshm\") pod \"8b71224c-5355-4319-a590-56105f625d2f\" (UID: \"8b71224c-5355-4319-a590-56105f625d2f\") "
Sep 14 13:26:42.032245 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032229    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-home\") pod \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") "
Sep 14 13:26:42.032674 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032258    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-dshm\") pod \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\" (UID: \"5c7d4e50-3882-439d-8202-bb7efdb6e8e9\") "
Sep 14 13:26:42.032674 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032350    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5c7d4e50-3882-439d-8202-bb7efdb6e8e9" (UID: "5c7d4e50-3882-439d-8202-bb7efdb6e8e9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.032779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032681    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-dshm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.032779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032702    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-tmp-dir\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.032779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032710    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-home\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.032779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032718    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-tmp-dir\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.032779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032727    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-home\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.032779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032738    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-home\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.032779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032751    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-tmp-dir\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.032779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032762    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55rf8\" (UniqueName: \"kubernetes.io/projected/f19605b1-bc2b-4a43-92a3-5a79e739ffab-kube-api-access-55rf8\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.032779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032776    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-model-cache\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.032779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032785    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-home\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.033355 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032792    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f19605b1-bc2b-4a43-92a3-5a79e739ffab-model-cache\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.033355 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032799    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f19605b1-bc2b-4a43-92a3-5a79e739ffab-tls-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.033355 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032806    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-model-cache\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.033355 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032893    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-model-cache" (OuterVolumeSpecName: "model-cache") pod "ef6e3627-df7d-404f-9e0c-a069dbc5d8f7" (UID: "ef6e3627-df7d-404f-9e0c-a069dbc5d8f7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.033355 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.032984    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8b71224c-5355-4319-a590-56105f625d2f" (UID: "8b71224c-5355-4319-a590-56105f625d2f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.033355 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.033125    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-model-cache" (OuterVolumeSpecName: "model-cache") pod "4cf9609a-f243-49b7-8847-3e7aaafb1d3c" (UID: "4cf9609a-f243-49b7-8847-3e7aaafb1d3c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.034207 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.033919    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-home" (OuterVolumeSpecName: "home") pod "5c7d4e50-3882-439d-8202-bb7efdb6e8e9" (UID: "5c7d4e50-3882-439d-8202-bb7efdb6e8e9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.034375 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.034289    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4cf9609a-f243-49b7-8847-3e7aaafb1d3c" (UID: "4cf9609a-f243-49b7-8847-3e7aaafb1d3c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.034843 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.034818    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-kube-api-access-pwgmn" (OuterVolumeSpecName: "kube-api-access-pwgmn") pod "ef6e3627-df7d-404f-9e0c-a069dbc5d8f7" (UID: "ef6e3627-df7d-404f-9e0c-a069dbc5d8f7"). InnerVolumeSpecName "kube-api-access-pwgmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:26:42.035458 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.035432    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b71224c-5355-4319-a590-56105f625d2f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8b71224c-5355-4319-a590-56105f625d2f" (UID: "8b71224c-5355-4319-a590-56105f625d2f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:26:42.036152 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.035974    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-dshm" (OuterVolumeSpecName: "dshm") pod "5c7d4e50-3882-439d-8202-bb7efdb6e8e9" (UID: "5c7d4e50-3882-439d-8202-bb7efdb6e8e9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.036152 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.036046    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4cf9609a-f243-49b7-8847-3e7aaafb1d3c" (UID: "4cf9609a-f243-49b7-8847-3e7aaafb1d3c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:26:42.036152 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.036100    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ef6e3627-df7d-404f-9e0c-a069dbc5d8f7" (UID: "ef6e3627-df7d-404f-9e0c-a069dbc5d8f7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:26:42.036152 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.036136    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5c7d4e50-3882-439d-8202-bb7efdb6e8e9" (UID: "5c7d4e50-3882-439d-8202-bb7efdb6e8e9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:26:42.036577 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.036558    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-dshm" (OuterVolumeSpecName: "dshm") pod "ef6e3627-df7d-404f-9e0c-a069dbc5d8f7" (UID: "ef6e3627-df7d-404f-9e0c-a069dbc5d8f7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.036665 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.036583    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-dshm" (OuterVolumeSpecName: "dshm") pod "8b71224c-5355-4319-a590-56105f625d2f" (UID: "8b71224c-5355-4319-a590-56105f625d2f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.036764 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.036747    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-kube-api-access-l7d22" (OuterVolumeSpecName: "kube-api-access-l7d22") pod "5c7d4e50-3882-439d-8202-bb7efdb6e8e9" (UID: "5c7d4e50-3882-439d-8202-bb7efdb6e8e9"). InnerVolumeSpecName "kube-api-access-l7d22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:26:42.037249 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.037228    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-dshm" (OuterVolumeSpecName: "dshm") pod "4cf9609a-f243-49b7-8847-3e7aaafb1d3c" (UID: "4cf9609a-f243-49b7-8847-3e7aaafb1d3c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:42.037318 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.037248    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-kube-api-access-lrhnz" (OuterVolumeSpecName: "kube-api-access-lrhnz") pod "4cf9609a-f243-49b7-8847-3e7aaafb1d3c" (UID: "4cf9609a-f243-49b7-8847-3e7aaafb1d3c"). InnerVolumeSpecName "kube-api-access-lrhnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:26:42.037576 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.037520    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b71224c-5355-4319-a590-56105f625d2f-kube-api-access-6hnzv" (OuterVolumeSpecName: "kube-api-access-6hnzv") pod "8b71224c-5355-4319-a590-56105f625d2f" (UID: "8b71224c-5355-4319-a590-56105f625d2f"). InnerVolumeSpecName "kube-api-access-6hnzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:26:42.134317 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134275    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-model-cache\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134317 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134309    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-tmp-dir\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134317 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134321    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-model-cache\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134334    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-dshm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134347    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hnzv\" (UniqueName: \"kubernetes.io/projected/8b71224c-5355-4319-a590-56105f625d2f-kube-api-access-6hnzv\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134359    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-dshm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134370    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-tmp-dir\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134379    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b71224c-5355-4319-a590-56105f625d2f-dshm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134388    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-home\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134398    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-dshm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134407    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b71224c-5355-4319-a590-56105f625d2f-tls-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134419    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-tls-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134430    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwgmn\" (UniqueName: \"kubernetes.io/projected/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7-kube-api-access-pwgmn\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134440    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrhnz\" (UniqueName: \"kubernetes.io/projected/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-kube-api-access-lrhnz\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134452    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf9609a-f243-49b7-8847-3e7aaafb1d3c-tls-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134464    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7d22\" (UniqueName: \"kubernetes.io/projected/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-kube-api-access-l7d22\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.134637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.134475    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c7d4e50-3882-439d-8202-bb7efdb6e8e9-tls-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:42.212248 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.212212    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"]
Sep 14 13:26:42.216203 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.216158    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:42.260859 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.260825    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"]
Sep 14 13:26:42.263358 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.263334    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/simulator-w0-bc851d-kserve-598f555d86-47dfw"]
Sep 14 13:26:42.273028 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.273002    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"]
Sep 14 13:26:42.276419 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.276399    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/simulator-w4-b3d6c5-kserve-fc46456db-kh4t4"]
Sep 14 13:26:42.286706 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.286643    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"]
Sep 14 13:26:42.292329 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.292307    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b3d6c5/distinct-2-w4-b3d6c5-kserve-555bdbc589-lmm2z"]
Sep 14 13:26:42.302319 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.302293    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"]
Sep 14 13:26:42.305917 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:42.305896    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-bc851d/distinct-w0-bc851d-kserve-6b59df5794-2xn9x"]
Sep 14 13:26:43.192976 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:43.192944    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:43.246727 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:43.246691    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cf9609a-f243-49b7-8847-3e7aaafb1d3c" path="/var/lib/kubelet/pods/4cf9609a-f243-49b7-8847-3e7aaafb1d3c/volumes"
Sep 14 13:26:43.247094 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:43.247081    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c7d4e50-3882-439d-8202-bb7efdb6e8e9" path="/var/lib/kubelet/pods/5c7d4e50-3882-439d-8202-bb7efdb6e8e9/volumes"
Sep 14 13:26:43.247433 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:43.247421    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b71224c-5355-4319-a590-56105f625d2f" path="/var/lib/kubelet/pods/8b71224c-5355-4319-a590-56105f625d2f/volumes"
Sep 14 13:26:43.248094 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:43.248076    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef6e3627-df7d-404f-9e0c-a069dbc5d8f7" path="/var/lib/kubelet/pods/ef6e3627-df7d-404f-9e0c-a069dbc5d8f7/volumes"
Sep 14 13:26:43.248439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:43.248428    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f19605b1-bc2b-4a43-92a3-5a79e739ffab" path="/var/lib/kubelet/pods/f19605b1-bc2b-4a43-92a3-5a79e739ffab/volumes"
Sep 14 13:26:43.293312 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:43.293278    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:43.533744 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:43.533665    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:44.037211 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:44.037173    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:44.045598 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:44.045565    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b3d6c5-29823195-j6brt"]
Sep 14 13:26:44.050085 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:44.050053    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-b3d6c5-29823195-j6brt"]
Sep 14 13:26:44.113879 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:44.113848    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:44.182418 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:44.182382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:44.282184 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:44.282152    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:44.557642 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:44.557608    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:44.654244 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:44.654217    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:45.246011 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:45.245974    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4aae587-eeea-4aa8-a45a-4255945c1ede" path="/var/lib/kubelet/pods/a4aae587-eeea-4aa8-a45a-4255945c1ede/volumes"
Sep 14 13:26:45.437942 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:45.437916    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:46.058780 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:46.058750    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:46.825125 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:46.825093    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:47.527059 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:47.527025    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:51.271664 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:51.271576    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:51.332223 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:51.332192    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:51.652964 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:51.652927    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:52.572160 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:52.572130    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:52.650115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:52.650052    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:52.828475 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:52.828398    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:55.709969 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:55.709928    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr" podUID="a1e51dfb-f589-49c9-bc1b-90293f6d1b2d" containerName="main" containerID="cri-o://7309c8696c7ede2c6baca3c2047b890a0701d44f9dee5639712d4862c562a8e4" gracePeriod=45
Sep 14 13:26:55.969990 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:55.969925    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:26:56.022643 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.022613    2581 generic.go:356] "Generic (PLEG): container finished" podID="a1e51dfb-f589-49c9-bc1b-90293f6d1b2d" containerID="7309c8696c7ede2c6baca3c2047b890a0701d44f9dee5639712d4862c562a8e4" exitCode=0
Sep 14 13:26:56.022798 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.022657    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"
Sep 14 13:26:56.022798 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.022696    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr" event={"ID":"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d","Type":"ContainerDied","Data":"7309c8696c7ede2c6baca3c2047b890a0701d44f9dee5639712d4862c562a8e4"}
Sep 14 13:26:56.022798 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.022741    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr" event={"ID":"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d","Type":"ContainerDied","Data":"47ab5a4e44c89625d8b332b3fa00285e77457a818d423469d1dd00ab6ee6b995"}
Sep 14 13:26:56.022798 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.022762    2581 scope.go:117] "RemoveContainer" containerID="7309c8696c7ede2c6baca3c2047b890a0701d44f9dee5639712d4862c562a8e4"
Sep 14 13:26:56.031346 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.031321    2581 scope.go:117] "RemoveContainer" containerID="7309c8696c7ede2c6baca3c2047b890a0701d44f9dee5639712d4862c562a8e4"
Sep 14 13:26:56.031655 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:26:56.031636    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7309c8696c7ede2c6baca3c2047b890a0701d44f9dee5639712d4862c562a8e4\": container with ID starting with 7309c8696c7ede2c6baca3c2047b890a0701d44f9dee5639712d4862c562a8e4 not found: ID does not exist" containerID="7309c8696c7ede2c6baca3c2047b890a0701d44f9dee5639712d4862c562a8e4"
Sep 14 13:26:56.031724 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.031663    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7309c8696c7ede2c6baca3c2047b890a0701d44f9dee5639712d4862c562a8e4"} err="failed to get container status \"7309c8696c7ede2c6baca3c2047b890a0701d44f9dee5639712d4862c562a8e4\": rpc error: code = NotFound desc = could not find container \"7309c8696c7ede2c6baca3c2047b890a0701d44f9dee5639712d4862c562a8e4\": container with ID starting with 7309c8696c7ede2c6baca3c2047b890a0701d44f9dee5639712d4862c562a8e4 not found: ID does not exist"
Sep 14 13:26:56.073291 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.073259    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-dshm\") pod \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") "
Sep 14 13:26:56.073433 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.073298    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsmmq\" (UniqueName: \"kubernetes.io/projected/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-kube-api-access-xsmmq\") pod \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") "
Sep 14 13:26:56.073433 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.073319    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-tmp-dir\") pod \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") "
Sep 14 13:26:56.073433 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.073365    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-tls-certs\") pod \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") "
Sep 14 13:26:56.073433 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.073405    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-home\") pod \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") "
Sep 14 13:26:56.073747 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.073551    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-model-cache\") pod \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\" (UID: \"a1e51dfb-f589-49c9-bc1b-90293f6d1b2d\") "
Sep 14 13:26:56.073747 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.073689    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a1e51dfb-f589-49c9-bc1b-90293f6d1b2d" (UID: "a1e51dfb-f589-49c9-bc1b-90293f6d1b2d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:56.073882 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.073852    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-model-cache" (OuterVolumeSpecName: "model-cache") pod "a1e51dfb-f589-49c9-bc1b-90293f6d1b2d" (UID: "a1e51dfb-f589-49c9-bc1b-90293f6d1b2d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:56.073939 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.073913    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-home" (OuterVolumeSpecName: "home") pod "a1e51dfb-f589-49c9-bc1b-90293f6d1b2d" (UID: "a1e51dfb-f589-49c9-bc1b-90293f6d1b2d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:56.073939 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.073921    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-tmp-dir\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:56.075715 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.075686    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-dshm" (OuterVolumeSpecName: "dshm") pod "a1e51dfb-f589-49c9-bc1b-90293f6d1b2d" (UID: "a1e51dfb-f589-49c9-bc1b-90293f6d1b2d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:56.075715 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.075690    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a1e51dfb-f589-49c9-bc1b-90293f6d1b2d" (UID: "a1e51dfb-f589-49c9-bc1b-90293f6d1b2d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:26:56.075867 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.075751    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-kube-api-access-xsmmq" (OuterVolumeSpecName: "kube-api-access-xsmmq") pod "a1e51dfb-f589-49c9-bc1b-90293f6d1b2d" (UID: "a1e51dfb-f589-49c9-bc1b-90293f6d1b2d"). InnerVolumeSpecName "kube-api-access-xsmmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:26:56.174725 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.174671    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-model-cache\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:56.174725 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.174716    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-dshm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:56.174725 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.174726    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsmmq\" (UniqueName: \"kubernetes.io/projected/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-kube-api-access-xsmmq\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:56.174725 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.174736    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-tls-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:56.174725 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.174746    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d-home\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:56.351110 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.351070    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"]
Sep 14 13:26:56.355156 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:56.355118    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14f3af/auth-test-model-14f3af-kserve-7b7d95cb47-ncwcr"]
Sep 14 13:26:57.231781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.231717    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls" podUID="113015c3-fc40-4151-a86b-c57555eb75f5" containerName="main" containerID="cri-o://67d9bb4e2838949c581b7f6d8441525ca071a33775d255a5541e465865108d1d" gracePeriod=45
Sep 14 13:26:57.249584 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.249555    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1e51dfb-f589-49c9-bc1b-90293f6d1b2d" path="/var/lib/kubelet/pods/a1e51dfb-f589-49c9-bc1b-90293f6d1b2d/volumes"
Sep 14 13:26:57.476585 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.476554    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:26:57.589260 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.589167    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x45zp\" (UniqueName: \"kubernetes.io/projected/113015c3-fc40-4151-a86b-c57555eb75f5-kube-api-access-x45zp\") pod \"113015c3-fc40-4151-a86b-c57555eb75f5\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") "
Sep 14 13:26:57.589260 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.589232    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-dshm\") pod \"113015c3-fc40-4151-a86b-c57555eb75f5\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") "
Sep 14 13:26:57.589507 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.589269    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-tmp-dir\") pod \"113015c3-fc40-4151-a86b-c57555eb75f5\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") "
Sep 14 13:26:57.589507 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.589302    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-home\") pod \"113015c3-fc40-4151-a86b-c57555eb75f5\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") "
Sep 14 13:26:57.589507 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.589360    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-model-cache\") pod \"113015c3-fc40-4151-a86b-c57555eb75f5\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") "
Sep 14 13:26:57.589507 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.589426    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113015c3-fc40-4151-a86b-c57555eb75f5-tls-certs\") pod \"113015c3-fc40-4151-a86b-c57555eb75f5\" (UID: \"113015c3-fc40-4151-a86b-c57555eb75f5\") "
Sep 14 13:26:57.589760 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.589614    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "113015c3-fc40-4151-a86b-c57555eb75f5" (UID: "113015c3-fc40-4151-a86b-c57555eb75f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:57.589760 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.589657    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-home" (OuterVolumeSpecName: "home") pod "113015c3-fc40-4151-a86b-c57555eb75f5" (UID: "113015c3-fc40-4151-a86b-c57555eb75f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:57.589760 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.589745    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-model-cache" (OuterVolumeSpecName: "model-cache") pod "113015c3-fc40-4151-a86b-c57555eb75f5" (UID: "113015c3-fc40-4151-a86b-c57555eb75f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:57.589926 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.589810    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-tmp-dir\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:57.589926 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.589827    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-home\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:57.591571 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.591547    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/113015c3-fc40-4151-a86b-c57555eb75f5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "113015c3-fc40-4151-a86b-c57555eb75f5" (UID: "113015c3-fc40-4151-a86b-c57555eb75f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:26:57.591659 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.591589    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-dshm" (OuterVolumeSpecName: "dshm") pod "113015c3-fc40-4151-a86b-c57555eb75f5" (UID: "113015c3-fc40-4151-a86b-c57555eb75f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:57.591717 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.591672    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/113015c3-fc40-4151-a86b-c57555eb75f5-kube-api-access-x45zp" (OuterVolumeSpecName: "kube-api-access-x45zp") pod "113015c3-fc40-4151-a86b-c57555eb75f5" (UID: "113015c3-fc40-4151-a86b-c57555eb75f5"). InnerVolumeSpecName "kube-api-access-x45zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:26:57.690940 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.690900    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/113015c3-fc40-4151-a86b-c57555eb75f5-tls-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:57.690940 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.690934    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x45zp\" (UniqueName: \"kubernetes.io/projected/113015c3-fc40-4151-a86b-c57555eb75f5-kube-api-access-x45zp\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:57.690940 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.690944    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-dshm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:57.691181 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:57.690954    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/113015c3-fc40-4151-a86b-c57555eb75f5-model-cache\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:26:58.035847 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:58.035804    2581 generic.go:356] "Generic (PLEG): container finished" podID="113015c3-fc40-4151-a86b-c57555eb75f5" containerID="67d9bb4e2838949c581b7f6d8441525ca071a33775d255a5541e465865108d1d" exitCode=0
Sep 14 13:26:58.035847 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:58.035851    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"
Sep 14 13:26:58.036137 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:58.035892    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls" event={"ID":"113015c3-fc40-4151-a86b-c57555eb75f5","Type":"ContainerDied","Data":"67d9bb4e2838949c581b7f6d8441525ca071a33775d255a5541e465865108d1d"}
Sep 14 13:26:58.036137 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:58.035944    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls" event={"ID":"113015c3-fc40-4151-a86b-c57555eb75f5","Type":"ContainerDied","Data":"29337822a84d13ce3c360a78cf7d5693673b7f7d017d1f108e9b74779466f4b1"}
Sep 14 13:26:58.036137 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:58.035973    2581 scope.go:117] "RemoveContainer" containerID="67d9bb4e2838949c581b7f6d8441525ca071a33775d255a5541e465865108d1d"
Sep 14 13:26:58.045924 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:58.045906    2581 scope.go:117] "RemoveContainer" containerID="67d9bb4e2838949c581b7f6d8441525ca071a33775d255a5541e465865108d1d"
Sep 14 13:26:58.046249 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:26:58.046230    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67d9bb4e2838949c581b7f6d8441525ca071a33775d255a5541e465865108d1d\": container with ID starting with 67d9bb4e2838949c581b7f6d8441525ca071a33775d255a5541e465865108d1d not found: ID does not exist" containerID="67d9bb4e2838949c581b7f6d8441525ca071a33775d255a5541e465865108d1d"
Sep 14 13:26:58.046295 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:58.046261    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67d9bb4e2838949c581b7f6d8441525ca071a33775d255a5541e465865108d1d"} err="failed to get container status \"67d9bb4e2838949c581b7f6d8441525ca071a33775d255a5541e465865108d1d\": rpc error: code = NotFound desc = could not find container \"67d9bb4e2838949c581b7f6d8441525ca071a33775d255a5541e465865108d1d\": container with ID starting with 67d9bb4e2838949c581b7f6d8441525ca071a33775d255a5541e465865108d1d not found: ID does not exist"
Sep 14 13:26:58.060127 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:58.060090    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"]
Sep 14 13:26:58.065333 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:58.063690    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/auth-test-model-eff5e8-kserve-58dc8fcbc9-4bmls"]
Sep 14 13:26:59.245647 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:26:59.245610    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="113015c3-fc40-4151-a86b-c57555eb75f5" path="/var/lib/kubelet/pods/113015c3-fc40-4151-a86b-c57555eb75f5/volumes"
Sep 14 13:27:01.533318 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:01.533280    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:01.659686 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:01.659652    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:27:05.041852 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:05.041811    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:05.138297 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:05.138265    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:05.238336 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:05.238298    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:09.581695 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:09.581661    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:09.682776 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:09.682736    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:09.916348 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:09.916314    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:10.227295 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:10.227216    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:10.541160 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:10.541080    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:12.537197 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:12.537163    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:13.064115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:13.064078    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:13.151099 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:13.151044    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:13.401312 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:13.401280    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:14.180651 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:14.180622    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:15.411422 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:15.411388    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:16.229553 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:16.229501    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:17.058270 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.058234    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"]
Sep 14 13:27:17.058876 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.058860    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1e51dfb-f589-49c9-bc1b-90293f6d1b2d" containerName="main"
Sep 14 13:27:17.058942 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.058880    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1e51dfb-f589-49c9-bc1b-90293f6d1b2d" containerName="main"
Sep 14 13:27:17.058942 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.058912    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113015c3-fc40-4151-a86b-c57555eb75f5" containerName="main"
Sep 14 13:27:17.058942 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.058920    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="113015c3-fc40-4151-a86b-c57555eb75f5" containerName="main"
Sep 14 13:27:17.058942 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.058931    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef6e3627-df7d-404f-9e0c-a069dbc5d8f7" containerName="main"
Sep 14 13:27:17.058942 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.058939    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef6e3627-df7d-404f-9e0c-a069dbc5d8f7" containerName="main"
Sep 14 13:27:17.059136 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.058948    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b71224c-5355-4319-a590-56105f625d2f" containerName="main"
Sep 14 13:27:17.059136 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.058956    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b71224c-5355-4319-a590-56105f625d2f" containerName="main"
Sep 14 13:27:17.059136 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.058968    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c7d4e50-3882-439d-8202-bb7efdb6e8e9" containerName="main"
Sep 14 13:27:17.059136 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.058976    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c7d4e50-3882-439d-8202-bb7efdb6e8e9" containerName="main"
Sep 14 13:27:17.059136 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.058996    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f19605b1-bc2b-4a43-92a3-5a79e739ffab" containerName="main"
Sep 14 13:27:17.059136 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.059006    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f19605b1-bc2b-4a43-92a3-5a79e739ffab" containerName="main"
Sep 14 13:27:17.059136 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.059024    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cf9609a-f243-49b7-8847-3e7aaafb1d3c" containerName="main"
Sep 14 13:27:17.059136 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.059033    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cf9609a-f243-49b7-8847-3e7aaafb1d3c" containerName="main"
Sep 14 13:27:17.059136 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.059112    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cf9609a-f243-49b7-8847-3e7aaafb1d3c" containerName="main"
Sep 14 13:27:17.059136 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.059125    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f19605b1-bc2b-4a43-92a3-5a79e739ffab" containerName="main"
Sep 14 13:27:17.059136 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.059134    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1e51dfb-f589-49c9-bc1b-90293f6d1b2d" containerName="main"
Sep 14 13:27:17.059499 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.059143    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b71224c-5355-4319-a590-56105f625d2f" containerName="main"
Sep 14 13:27:17.059499 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.059152    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c7d4e50-3882-439d-8202-bb7efdb6e8e9" containerName="main"
Sep 14 13:27:17.059499 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.059161    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef6e3627-df7d-404f-9e0c-a069dbc5d8f7" containerName="main"
Sep 14 13:27:17.059499 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.059170    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="113015c3-fc40-4151-a86b-c57555eb75f5" containerName="main"
Sep 14 13:27:17.063233 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.063206    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.066216 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.066189    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-622b87-serving-cert\""
Sep 14 13:27:17.079252 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.079224    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"]
Sep 14 13:27:17.104724 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.104673    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:17.170830 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.170795    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-maas-api-tls\") pod \"maas-api-e2e-api-a-622b87-8675d577c6-vwtkb\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.171042 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.170865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-openshift-service-ca\") pod \"maas-api-e2e-api-a-622b87-8675d577c6-vwtkb\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.171042 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.170934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c47vg\" (UniqueName: \"kubernetes.io/projected/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-kube-api-access-c47vg\") pod \"maas-api-e2e-api-a-622b87-8675d577c6-vwtkb\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.171175 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.171062    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-metrics-certs\") pod \"maas-api-e2e-api-a-622b87-8675d577c6-vwtkb\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.224367 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.224332    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:17.271767 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.271725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-openshift-service-ca\") pod \"maas-api-e2e-api-a-622b87-8675d577c6-vwtkb\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.271973 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.271774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c47vg\" (UniqueName: \"kubernetes.io/projected/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-kube-api-access-c47vg\") pod \"maas-api-e2e-api-a-622b87-8675d577c6-vwtkb\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.271973 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.271815    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-metrics-certs\") pod \"maas-api-e2e-api-a-622b87-8675d577c6-vwtkb\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.271973 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.271900    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-maas-api-tls\") pod \"maas-api-e2e-api-a-622b87-8675d577c6-vwtkb\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.272144 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:27:17.272021    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-serving-cert: secret "maas-api-e2e-api-a-622b87-serving-cert" not found
Sep 14 13:27:17.272144 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:27:17.272083    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-maas-api-tls podName:40d1e5b8-8a69-4090-b2ba-87f00d11dd8c nodeName:}" failed. No retries permitted until 2026-09-14 13:27:17.772062371 +0000 UTC m=+1731.063396168 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-maas-api-tls") pod "maas-api-e2e-api-a-622b87-8675d577c6-vwtkb" (UID: "40d1e5b8-8a69-4090-b2ba-87f00d11dd8c") : secret "maas-api-e2e-api-a-622b87-serving-cert" not found
Sep 14 13:27:17.272467 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.272443    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-openshift-service-ca\") pod \"maas-api-e2e-api-a-622b87-8675d577c6-vwtkb\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.274313 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.274289    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-metrics-certs\") pod \"maas-api-e2e-api-a-622b87-8675d577c6-vwtkb\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.291025 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.290988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c47vg\" (UniqueName: \"kubernetes.io/projected/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-kube-api-access-c47vg\") pod \"maas-api-e2e-api-a-622b87-8675d577c6-vwtkb\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.468757 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.468718    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:17.664507 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.664465    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:17.760615 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.760514    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:17.776724 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.776690    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-maas-api-tls\") pod \"maas-api-e2e-api-a-622b87-8675d577c6-vwtkb\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.786627 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.780202    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-maas-api-tls\") pod \"maas-api-e2e-api-a-622b87-8675d577c6-vwtkb\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.794453 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.794420    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"]
Sep 14 13:27:17.798495 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.798466    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"
Sep 14 13:27:17.801486 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.801457    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-622b87\""
Sep 14 13:27:17.801665 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.801582    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-622b87-dockercfg-xnsn8\""
Sep 14 13:27:17.810427 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.810393    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"]
Sep 14 13:27:17.877703 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.877670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e673c82c-e41b-417a-bf11-0a9b3caef716-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt\" (UID: \"e673c82c-e41b-417a-bf11-0a9b3caef716\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"
Sep 14 13:27:17.877885 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.877717    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76r25\" (UniqueName: \"kubernetes.io/projected/e673c82c-e41b-417a-bf11-0a9b3caef716-kube-api-access-76r25\") pod \"payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt\" (UID: \"e673c82c-e41b-417a-bf11-0a9b3caef716\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"
Sep 14 13:27:17.973576 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.973519    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:17.978779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.978755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e673c82c-e41b-417a-bf11-0a9b3caef716-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt\" (UID: \"e673c82c-e41b-417a-bf11-0a9b3caef716\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"
Sep 14 13:27:17.978876 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.978798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76r25\" (UniqueName: \"kubernetes.io/projected/e673c82c-e41b-417a-bf11-0a9b3caef716-kube-api-access-76r25\") pod \"payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt\" (UID: \"e673c82c-e41b-417a-bf11-0a9b3caef716\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"
Sep 14 13:27:17.979458 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.979431    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e673c82c-e41b-417a-bf11-0a9b3caef716-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt\" (UID: \"e673c82c-e41b-417a-bf11-0a9b3caef716\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"
Sep 14 13:27:17.988304 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:17.988266    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76r25\" (UniqueName: \"kubernetes.io/projected/e673c82c-e41b-417a-bf11-0a9b3caef716-kube-api-access-76r25\") pod \"payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt\" (UID: \"e673c82c-e41b-417a-bf11-0a9b3caef716\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"
Sep 14 13:27:18.113144 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:18.113104    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"
Sep 14 13:27:18.256579 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:18.256552    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"]
Sep 14 13:27:18.258614 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:27:18.258582    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40d1e5b8_8a69_4090_b2ba_87f00d11dd8c.slice/crio-3cfa15c21962e84d25f4166dc279861a5f461a742b494135e5d95e286a6f313c WatchSource:0}: Error finding container 3cfa15c21962e84d25f4166dc279861a5f461a742b494135e5d95e286a6f313c: Status 404 returned error can't find the container with id 3cfa15c21962e84d25f4166dc279861a5f461a742b494135e5d95e286a6f313c
Sep 14 13:27:18.259932 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:18.259914    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 13:27:18.389779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:18.389752    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"]
Sep 14 13:27:18.391666 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:27:18.391640    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode673c82c_e41b_417a_bf11_0a9b3caef716.slice/crio-7e0363359ae89fe639124d46acb4aa41f4aa8f9d4c7aec8d381fb9e26e03fd67 WatchSource:0}: Error finding container 7e0363359ae89fe639124d46acb4aa41f4aa8f9d4c7aec8d381fb9e26e03fd67: Status 404 returned error can't find the container with id 7e0363359ae89fe639124d46acb4aa41f4aa8f9d4c7aec8d381fb9e26e03fd67
Sep 14 13:27:19.152988 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:19.152953    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb" event={"ID":"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c","Type":"ContainerStarted","Data":"335ca64730b07cb0c8f3374f23c0c974d8f5ceb296e15629c498ed6d3254f06e"}
Sep 14 13:27:19.153476 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:19.153022    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:19.153476 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:19.153048    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb" event={"ID":"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c","Type":"ContainerStarted","Data":"3cfa15c21962e84d25f4166dc279861a5f461a742b494135e5d95e286a6f313c"}
Sep 14 13:27:19.154663 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:19.154626    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt" event={"ID":"e673c82c-e41b-417a-bf11-0a9b3caef716","Type":"ContainerStarted","Data":"8ef4d7295f80b6bccbe5aa108d8d18d3bf56372585ae21793b502fdc3c8d29bd"}
Sep 14 13:27:19.154791 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:19.154677    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt" event={"ID":"e673c82c-e41b-417a-bf11-0a9b3caef716","Type":"ContainerStarted","Data":"7e0363359ae89fe639124d46acb4aa41f4aa8f9d4c7aec8d381fb9e26e03fd67"}
Sep 14 13:27:19.154791 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:19.154723    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"
Sep 14 13:27:19.175950 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:19.175901    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb" podStartSLOduration=1.7852977509999999 podStartE2EDuration="2.175885837s" totalImagesPullingTime="390.588086ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:27:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:27:19.173891485 +0000 UTC m=+1732.465225328" watchObservedRunningTime="2026-09-14 13:27:19.175885837 +0000 UTC m=+1732.467219653"
Sep 14 13:27:19.200624 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:19.200563    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt" podStartSLOduration=2.200542906 podStartE2EDuration="2.200542906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:27:19.197510022 +0000 UTC m=+1732.488843840" watchObservedRunningTime="2026-09-14 13:27:19.200542906 +0000 UTC m=+1732.491876716"
Sep 14 13:27:22.680727 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:22.680688    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:27:25.166980 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.166952    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:27:25.723483 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.723446    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"]
Sep 14 13:27:25.727523 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.727494    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.730139 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.730115    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14f3af\"/\"default-dockercfg-pbl7q\""
Sep 14 13:27:25.730696 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.730664    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-14f3af\"/\"openshift-service-ca.crt\""
Sep 14 13:27:25.730933 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.730913    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14f3af\"/\"sub-test-model-14f3af-kserve-self-signed-certs\""
Sep 14 13:27:25.731194 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.731172    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-14f3af\"/\"kube-root-ca.crt\""
Sep 14 13:27:25.737433 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.737404    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"]
Sep 14 13:27:25.858981 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.858939    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509599fa-9990-4da1-ab73-9439d03a5f28-tls-certs\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.859162 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.858993    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-home\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.859162 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.859049    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-dshm\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.859162 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.859069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xsmx\" (UniqueName: \"kubernetes.io/projected/509599fa-9990-4da1-ab73-9439d03a5f28-kube-api-access-2xsmx\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.859162 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.859122    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-tmp-dir\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.859162 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.859160    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-model-cache\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.960408 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.960371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-model-cache\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.960647 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.960450    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509599fa-9990-4da1-ab73-9439d03a5f28-tls-certs\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.960647 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.960478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-home\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.960647 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.960511    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-dshm\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.960647 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.960562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xsmx\" (UniqueName: \"kubernetes.io/projected/509599fa-9990-4da1-ab73-9439d03a5f28-kube-api-access-2xsmx\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.960647 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.960596    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-tmp-dir\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.960920 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.960897    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-home\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.960920 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.960910    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-model-cache\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.961013 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.960961    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-tmp-dir\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.962836 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.962816    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-dshm\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.963164 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.963145    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509599fa-9990-4da1-ab73-9439d03a5f28-tls-certs\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:25.970229 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:25.970203    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xsmx\" (UniqueName: \"kubernetes.io/projected/509599fa-9990-4da1-ab73-9439d03a5f28-kube-api-access-2xsmx\") pod \"sub-test-model-14f3af-kserve-75b49459b9-ztzx8\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") " pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:26.026449 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:26.026361    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:26.039485 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:26.039452    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:26.316739 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:26.316712    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"]
Sep 14 13:27:26.318800 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:27:26.318763    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod509599fa_9990_4da1_ab73_9439d03a5f28.slice/crio-b00bed50ceb337ad17ae3d593679b140773948de5e910fe02e4034ddcf96841f WatchSource:0}: Error finding container b00bed50ceb337ad17ae3d593679b140773948de5e910fe02e4034ddcf96841f: Status 404 returned error can't find the container with id b00bed50ceb337ad17ae3d593679b140773948de5e910fe02e4034ddcf96841f
Sep 14 13:27:27.198586 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:27.198523    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8" event={"ID":"509599fa-9990-4da1-ab73-9439d03a5f28","Type":"ContainerStarted","Data":"e18498437619ca6fe11af310f6d9214e0a0994dd15fc4fe9493e40b2ccd65b95"}
Sep 14 13:27:27.198586 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:27.198593    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8" event={"ID":"509599fa-9990-4da1-ab73-9439d03a5f28","Type":"ContainerStarted","Data":"b00bed50ceb337ad17ae3d593679b140773948de5e910fe02e4034ddcf96841f"}
Sep 14 13:27:27.219026 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:27.218967    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8" podStartSLOduration=2.218951946 podStartE2EDuration="2.218951946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:27:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:27:27.217283732 +0000 UTC m=+1740.508617553" watchObservedRunningTime="2026-09-14 13:27:27.218951946 +0000 UTC m=+1740.510285818"
Sep 14 13:27:27.436821 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:27.436790    2581 scope.go:117] "RemoveContainer" containerID="a4145e2cfec6fcebdc794ba9713ffdfe5727d5f4658bc080e44e1464825f55d8"
Sep 14 13:27:30.163715 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:30.163682    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"
Sep 14 13:27:36.040288 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:36.040243    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:36.040751 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:36.040313    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:36.052928 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:36.052892    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:36.258345 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:36.258313    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:27:38.325091 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:38.325057    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:38.956973 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:38.956939    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:27:39.372228 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:39.372195    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:40.103739 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:40.103706    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:40.839375 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:40.839342    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:41.514387 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:41.514356    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:42.159136 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:42.159105    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:42.944851 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:42.944815    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:43.096544 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:43.096500    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:43.524071 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:43.524035    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:43.582837 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:43.582804    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:43.850453 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:43.850372    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:44.002884 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:44.002855    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:46.537406 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:46.537373    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:47.264234 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:47.264193    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:47.349821 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:47.349783    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:48.192339 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:48.192305    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:48.955680 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:48.955634    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:49.648854 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:49.648808    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:50.405651 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:50.405614    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:51.043452 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.043423    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:51.143635 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.143607    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:51.348431 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.348348    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:51.568349 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.568313    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:51.668463 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.668429    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:51.684921 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.684889    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"]
Sep 14 13:27:51.689035 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.689014    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"
Sep 14 13:27:51.691734 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.691714    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-1cf25e-dockercfg-mdzkj\""
Sep 14 13:27:51.691734 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.691727    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-1cf25e\""
Sep 14 13:27:51.698411 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.698389    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"]
Sep 14 13:27:51.701438 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.701411    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bf768c3-5645-44bc-82e4-9a4b04e87f13-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv\" (UID: \"2bf768c3-5645-44bc-82e4-9a4b04e87f13\") " pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"
Sep 14 13:27:51.701584 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.701546    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9fww\" (UniqueName: \"kubernetes.io/projected/2bf768c3-5645-44bc-82e4-9a4b04e87f13-kube-api-access-x9fww\") pod \"payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv\" (UID: \"2bf768c3-5645-44bc-82e4-9a4b04e87f13\") " pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"
Sep 14 13:27:51.802895 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.802855    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9fww\" (UniqueName: \"kubernetes.io/projected/2bf768c3-5645-44bc-82e4-9a4b04e87f13-kube-api-access-x9fww\") pod \"payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv\" (UID: \"2bf768c3-5645-44bc-82e4-9a4b04e87f13\") " pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"
Sep 14 13:27:51.803075 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.802947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bf768c3-5645-44bc-82e4-9a4b04e87f13-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv\" (UID: \"2bf768c3-5645-44bc-82e4-9a4b04e87f13\") " pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"
Sep 14 13:27:51.803658 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.803640    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bf768c3-5645-44bc-82e4-9a4b04e87f13-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv\" (UID: \"2bf768c3-5645-44bc-82e4-9a4b04e87f13\") " pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"
Sep 14 13:27:51.812495 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:51.812461    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9fww\" (UniqueName: \"kubernetes.io/projected/2bf768c3-5645-44bc-82e4-9a4b04e87f13-kube-api-access-x9fww\") pod \"payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv\" (UID: \"2bf768c3-5645-44bc-82e4-9a4b04e87f13\") " pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"
Sep 14 13:27:52.001069 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:52.000979    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"
Sep 14 13:27:52.274576 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:52.274552    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"]
Sep 14 13:27:52.277099 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:27:52.277070    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bf768c3_5645_44bc_82e4_9a4b04e87f13.slice/crio-b6c5894d4fe417a45f1d555991c9b0e32768fca41c1500f4967ad9fa73ea3c6b WatchSource:0}: Error finding container b6c5894d4fe417a45f1d555991c9b0e32768fca41c1500f4967ad9fa73ea3c6b: Status 404 returned error can't find the container with id b6c5894d4fe417a45f1d555991c9b0e32768fca41c1500f4967ad9fa73ea3c6b
Sep 14 13:27:52.330118 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:52.330092    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv" event={"ID":"2bf768c3-5645-44bc-82e4-9a4b04e87f13","Type":"ContainerStarted","Data":"b6c5894d4fe417a45f1d555991c9b0e32768fca41c1500f4967ad9fa73ea3c6b"}
Sep 14 13:27:53.342481 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:53.342443    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv" event={"ID":"2bf768c3-5645-44bc-82e4-9a4b04e87f13","Type":"ContainerStarted","Data":"27ba31974cc83b1a59f67ed897d05970a7eb55b40098dd09c79f5cdb0e62bec7"}
Sep 14 13:27:53.342925 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:53.342572    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"
Sep 14 13:27:55.364856 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:55.364801    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv" podStartSLOduration=4.364782784 podStartE2EDuration="4.364782784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:27:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:27:53.362601306 +0000 UTC m=+1766.653935122" watchObservedRunningTime="2026-09-14 13:27:55.364782784 +0000 UTC m=+1768.656116598"
Sep 14 13:27:55.365624 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:27:55.365605    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:28:04.351573 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:04.351542    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"
Sep 14 13:28:12.632840 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:12.632800    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:13.457315 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:13.457280    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:14.118700 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:14.118664    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:14.762656 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:14.762619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:15.597622 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:15.597588    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:16.225069 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.225032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:16.412195 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.412164    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"]
Sep 14 13:28:16.416910 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.416881    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.419733 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.419708    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-eff5e8\"/\"kube-root-ca.crt\""
Sep 14 13:28:16.420894 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.420869    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-eff5e8\"/\"default-dockercfg-hhsnb\""
Sep 14 13:28:16.421021 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.420869    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-eff5e8\"/\"openshift-service-ca.crt\""
Sep 14 13:28:16.421021 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.420968    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-eff5e8\"/\"sub-test-model-eff5e8-kserve-self-signed-certs\""
Sep 14 13:28:16.425941 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.425920    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"]
Sep 14 13:28:16.437994 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.437969    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:16.552294 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.552203    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwrkm\" (UniqueName: \"kubernetes.io/projected/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-kube-api-access-vwrkm\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.552294 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.552278    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-dshm\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.552487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.552303    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-home\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.552487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.552329    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-model-cache\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.552487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.552433    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-tmp-dir\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.552630 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.552506    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-tls-certs\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.653150 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.653114    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-model-cache\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.653626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.653171    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-tmp-dir\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.653626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.653233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-tls-certs\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.653626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.653270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwrkm\" (UniqueName: \"kubernetes.io/projected/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-kube-api-access-vwrkm\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.653626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.653342    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-dshm\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.653626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.653376    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-home\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.654093 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.653626    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-model-cache\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.654093 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.653634    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-tmp-dir\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.654093 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.653691    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-home\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.655593 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.655567    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-dshm\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.655889 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.655871    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-tls-certs\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.661835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.661806    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwrkm\" (UniqueName: \"kubernetes.io/projected/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-kube-api-access-vwrkm\") pod \"sub-test-model-eff5e8-kserve-79bb994c8f-9tld6\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:16.726940 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:16.726904    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:17.007634 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:17.007605    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"]
Sep 14 13:28:17.009094 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:28:17.009064    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd40f778_7b9c_44e3_b84a_d96fb071cfd2.slice/crio-69ca213778453e5352d7e2b18d2bf32ab73d8c2b1bcd0fca48b9c5ddcf6948c6 WatchSource:0}: Error finding container 69ca213778453e5352d7e2b18d2bf32ab73d8c2b1bcd0fca48b9c5ddcf6948c6: Status 404 returned error can't find the container with id 69ca213778453e5352d7e2b18d2bf32ab73d8c2b1bcd0fca48b9c5ddcf6948c6
Sep 14 13:28:17.117610 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:17.117563    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:17.380670 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:17.380638    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:17.470229 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:17.470194    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6" event={"ID":"bd40f778-7b9c-44e3-b84a-d96fb071cfd2","Type":"ContainerStarted","Data":"8f732d36465bbf053de845224532ac6f2c6855222f15888c3cd38f53522cd715"}
Sep 14 13:28:17.470229 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:17.470244    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6" event={"ID":"bd40f778-7b9c-44e3-b84a-d96fb071cfd2","Type":"ContainerStarted","Data":"69ca213778453e5352d7e2b18d2bf32ab73d8c2b1bcd0fca48b9c5ddcf6948c6"}
Sep 14 13:28:17.489775 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:17.489726    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6" podStartSLOduration=1.489709727 podStartE2EDuration="1.489709727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:28:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:28:17.488582808 +0000 UTC m=+1790.779916625" watchObservedRunningTime="2026-09-14 13:28:17.489709727 +0000 UTC m=+1790.781043544"
Sep 14 13:28:17.844145 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:17.844025    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:20.714623 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:20.714512    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:21.366568 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:21.366514    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:21.662917 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:21.662825    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:28:22.169373 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:22.169341    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:23.009623 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:23.009585    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:23.924653 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:23.924615    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:24.634581 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:24.634550    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:25.443597 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:25.443571    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:26.148874 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:26.148827    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:26.727919 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:26.727876    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:26.728331 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:26.727935    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:26.741020 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:26.740991    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:27.288005 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:27.287973    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:28:27.291482 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:27.291448    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:28:27.537331 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:27.537293    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:28:35.876962 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:35.876928    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"]
Sep 14 13:28:35.877396 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:35.877128    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb" podUID="40d1e5b8-8a69-4090-b2ba-87f00d11dd8c" containerName="maas-api" containerID="cri-o://335ca64730b07cb0c8f3374f23c0c974d8f5ceb296e15629c498ed6d3254f06e" gracePeriod=30
Sep 14 13:28:35.893254 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:35.893215    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:35.964408 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:35.964367    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"]
Sep 14 13:28:35.964749 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:35.964715    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt" podUID="e673c82c-e41b-417a-bf11-0a9b3caef716" containerName="payload-pre-processing" containerID="cri-o://8ef4d7295f80b6bccbe5aa108d8d18d3bf56372585ae21793b502fdc3c8d29bd" gracePeriod=30
Sep 14 13:28:35.993837 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:35.993788    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:36.092316 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.092281    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:36.172400 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.172369    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:36.229288 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.229259    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:28:36.258662 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.258634    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"
Sep 14 13:28:36.350744 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.350710    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-maas-api-tls\") pod \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") "
Sep 14 13:28:36.350744 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.350748    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c47vg\" (UniqueName: \"kubernetes.io/projected/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-kube-api-access-c47vg\") pod \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") "
Sep 14 13:28:36.350956 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.350767    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-metrics-certs\") pod \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") "
Sep 14 13:28:36.350956 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.350794    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e673c82c-e41b-417a-bf11-0a9b3caef716-plugins-config-volume\") pod \"e673c82c-e41b-417a-bf11-0a9b3caef716\" (UID: \"e673c82c-e41b-417a-bf11-0a9b3caef716\") "
Sep 14 13:28:36.350956 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.350889    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76r25\" (UniqueName: \"kubernetes.io/projected/e673c82c-e41b-417a-bf11-0a9b3caef716-kube-api-access-76r25\") pod \"e673c82c-e41b-417a-bf11-0a9b3caef716\" (UID: \"e673c82c-e41b-417a-bf11-0a9b3caef716\") "
Sep 14 13:28:36.350956 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.350935    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-openshift-service-ca\") pod \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\" (UID: \"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c\") "
Sep 14 13:28:36.351375 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.351339    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e673c82c-e41b-417a-bf11-0a9b3caef716-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e673c82c-e41b-417a-bf11-0a9b3caef716" (UID: "e673c82c-e41b-417a-bf11-0a9b3caef716"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:28:36.351485 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.351376    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "40d1e5b8-8a69-4090-b2ba-87f00d11dd8c" (UID: "40d1e5b8-8a69-4090-b2ba-87f00d11dd8c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:28:36.353053 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.353023    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "40d1e5b8-8a69-4090-b2ba-87f00d11dd8c" (UID: "40d1e5b8-8a69-4090-b2ba-87f00d11dd8c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:28:36.353144 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.353049    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-kube-api-access-c47vg" (OuterVolumeSpecName: "kube-api-access-c47vg") pod "40d1e5b8-8a69-4090-b2ba-87f00d11dd8c" (UID: "40d1e5b8-8a69-4090-b2ba-87f00d11dd8c"). InnerVolumeSpecName "kube-api-access-c47vg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:28:36.353144 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.353069    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "40d1e5b8-8a69-4090-b2ba-87f00d11dd8c" (UID: "40d1e5b8-8a69-4090-b2ba-87f00d11dd8c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:28:36.353144 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.353076    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e673c82c-e41b-417a-bf11-0a9b3caef716-kube-api-access-76r25" (OuterVolumeSpecName: "kube-api-access-76r25") pod "e673c82c-e41b-417a-bf11-0a9b3caef716" (UID: "e673c82c-e41b-417a-bf11-0a9b3caef716"). InnerVolumeSpecName "kube-api-access-76r25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:28:36.452232 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.452146    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-maas-api-tls\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:28:36.452232 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.452176    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c47vg\" (UniqueName: \"kubernetes.io/projected/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-kube-api-access-c47vg\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:28:36.452232 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.452188    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-metrics-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:28:36.452232 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.452197    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e673c82c-e41b-417a-bf11-0a9b3caef716-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:28:36.452232 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.452205    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76r25\" (UniqueName: \"kubernetes.io/projected/e673c82c-e41b-417a-bf11-0a9b3caef716-kube-api-access-76r25\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:28:36.452232 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.452213    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c-openshift-service-ca\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:28:36.539147 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.539094    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:36.576361 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.576325    2581 generic.go:356] "Generic (PLEG): container finished" podID="40d1e5b8-8a69-4090-b2ba-87f00d11dd8c" containerID="335ca64730b07cb0c8f3374f23c0c974d8f5ceb296e15629c498ed6d3254f06e" exitCode=0
Sep 14 13:28:36.576556 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.576372    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"
Sep 14 13:28:36.576556 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.576418    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb" event={"ID":"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c","Type":"ContainerDied","Data":"335ca64730b07cb0c8f3374f23c0c974d8f5ceb296e15629c498ed6d3254f06e"}
Sep 14 13:28:36.576556 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.576474    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb" event={"ID":"40d1e5b8-8a69-4090-b2ba-87f00d11dd8c","Type":"ContainerDied","Data":"3cfa15c21962e84d25f4166dc279861a5f461a742b494135e5d95e286a6f313c"}
Sep 14 13:28:36.576556 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.576501    2581 scope.go:117] "RemoveContainer" containerID="335ca64730b07cb0c8f3374f23c0c974d8f5ceb296e15629c498ed6d3254f06e"
Sep 14 13:28:36.577779 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.577748    2581 generic.go:356] "Generic (PLEG): container finished" podID="e673c82c-e41b-417a-bf11-0a9b3caef716" containerID="8ef4d7295f80b6bccbe5aa108d8d18d3bf56372585ae21793b502fdc3c8d29bd" exitCode=0
Sep 14 13:28:36.577902 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.577807    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"
Sep 14 13:28:36.577902 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.577835    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt" event={"ID":"e673c82c-e41b-417a-bf11-0a9b3caef716","Type":"ContainerDied","Data":"8ef4d7295f80b6bccbe5aa108d8d18d3bf56372585ae21793b502fdc3c8d29bd"}
Sep 14 13:28:36.577902 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.577891    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt" event={"ID":"e673c82c-e41b-417a-bf11-0a9b3caef716","Type":"ContainerDied","Data":"7e0363359ae89fe639124d46acb4aa41f4aa8f9d4c7aec8d381fb9e26e03fd67"}
Sep 14 13:28:36.585759 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.585464    2581 scope.go:117] "RemoveContainer" containerID="335ca64730b07cb0c8f3374f23c0c974d8f5ceb296e15629c498ed6d3254f06e"
Sep 14 13:28:36.585836 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:28:36.585799    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"335ca64730b07cb0c8f3374f23c0c974d8f5ceb296e15629c498ed6d3254f06e\": container with ID starting with 335ca64730b07cb0c8f3374f23c0c974d8f5ceb296e15629c498ed6d3254f06e not found: ID does not exist" containerID="335ca64730b07cb0c8f3374f23c0c974d8f5ceb296e15629c498ed6d3254f06e"
Sep 14 13:28:36.585836 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.585822    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"335ca64730b07cb0c8f3374f23c0c974d8f5ceb296e15629c498ed6d3254f06e"} err="failed to get container status \"335ca64730b07cb0c8f3374f23c0c974d8f5ceb296e15629c498ed6d3254f06e\": rpc error: code = NotFound desc = could not find container \"335ca64730b07cb0c8f3374f23c0c974d8f5ceb296e15629c498ed6d3254f06e\": container with ID starting with 335ca64730b07cb0c8f3374f23c0c974d8f5ceb296e15629c498ed6d3254f06e not found: ID does not exist"
Sep 14 13:28:36.585957 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.585838    2581 scope.go:117] "RemoveContainer" containerID="8ef4d7295f80b6bccbe5aa108d8d18d3bf56372585ae21793b502fdc3c8d29bd"
Sep 14 13:28:36.594052 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.594034    2581 scope.go:117] "RemoveContainer" containerID="8ef4d7295f80b6bccbe5aa108d8d18d3bf56372585ae21793b502fdc3c8d29bd"
Sep 14 13:28:36.594358 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:28:36.594339    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ef4d7295f80b6bccbe5aa108d8d18d3bf56372585ae21793b502fdc3c8d29bd\": container with ID starting with 8ef4d7295f80b6bccbe5aa108d8d18d3bf56372585ae21793b502fdc3c8d29bd not found: ID does not exist" containerID="8ef4d7295f80b6bccbe5aa108d8d18d3bf56372585ae21793b502fdc3c8d29bd"
Sep 14 13:28:36.594404 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.594364    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ef4d7295f80b6bccbe5aa108d8d18d3bf56372585ae21793b502fdc3c8d29bd"} err="failed to get container status \"8ef4d7295f80b6bccbe5aa108d8d18d3bf56372585ae21793b502fdc3c8d29bd\": rpc error: code = NotFound desc = could not find container \"8ef4d7295f80b6bccbe5aa108d8d18d3bf56372585ae21793b502fdc3c8d29bd\": container with ID starting with 8ef4d7295f80b6bccbe5aa108d8d18d3bf56372585ae21793b502fdc3c8d29bd not found: ID does not exist"
Sep 14 13:28:36.607578 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.607551    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"]
Sep 14 13:28:36.611838 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.611811    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-622b87-8675d577c6-vwtkb"]
Sep 14 13:28:36.625228 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.625199    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"]
Sep 14 13:28:36.630835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.630814    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-622b87-784b8d8f78-vx9qt"]
Sep 14 13:28:36.631268 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:36.631252    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:37.245992 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:37.245909    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40d1e5b8-8a69-4090-b2ba-87f00d11dd8c" path="/var/lib/kubelet/pods/40d1e5b8-8a69-4090-b2ba-87f00d11dd8c/volumes"
Sep 14 13:28:37.246366 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:37.246309    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e673c82c-e41b-417a-bf11-0a9b3caef716" path="/var/lib/kubelet/pods/e673c82c-e41b-417a-bf11-0a9b3caef716/volumes"
Sep 14 13:28:37.359008 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:37.358978    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:38.177975 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:38.177937    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:38.941989 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:38.941953    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:39.551984 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:39.551957    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:40.252935 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:40.252902    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:41.057949 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:41.057914    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:28:43.546900 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:43.546863    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:43.642992 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:43.642958    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:43.817378 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:43.817281    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:44.431256 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:44.431220    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:45.233115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:45.233086    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:45.863995 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:45.863949    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:46.779381 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:46.779347    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:47.626889 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:47.626842    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:47.830716 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:47.830675    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:48.454131 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:48.454098    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:49.169590 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:49.169553    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:50.230746 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:50.230712    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:54.398179 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:28:54.398142    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:00.014061 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.014032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:00.046818 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.046782    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"]
Sep 14 13:29:00.047023 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.047002    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv" podUID="2bf768c3-5645-44bc-82e4-9a4b04e87f13" containerName="payload-processing" containerID="cri-o://27ba31974cc83b1a59f67ed897d05970a7eb55b40098dd09c79f5cdb0e62bec7" gracePeriod=30
Sep 14 13:29:00.113979 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.113947    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:00.193305 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.193260    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:00.256176 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.256140    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:00.312033 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.312005    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"
Sep 14 13:29:00.380820 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.380786    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9fww\" (UniqueName: \"kubernetes.io/projected/2bf768c3-5645-44bc-82e4-9a4b04e87f13-kube-api-access-x9fww\") pod \"2bf768c3-5645-44bc-82e4-9a4b04e87f13\" (UID: \"2bf768c3-5645-44bc-82e4-9a4b04e87f13\") "
Sep 14 13:29:00.381031 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.380882    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bf768c3-5645-44bc-82e4-9a4b04e87f13-plugins-config-volume\") pod \"2bf768c3-5645-44bc-82e4-9a4b04e87f13\" (UID: \"2bf768c3-5645-44bc-82e4-9a4b04e87f13\") "
Sep 14 13:29:00.381262 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.381235    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bf768c3-5645-44bc-82e4-9a4b04e87f13-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2bf768c3-5645-44bc-82e4-9a4b04e87f13" (UID: "2bf768c3-5645-44bc-82e4-9a4b04e87f13"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:29:00.383066 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.383036    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bf768c3-5645-44bc-82e4-9a4b04e87f13-kube-api-access-x9fww" (OuterVolumeSpecName: "kube-api-access-x9fww") pod "2bf768c3-5645-44bc-82e4-9a4b04e87f13" (UID: "2bf768c3-5645-44bc-82e4-9a4b04e87f13"). InnerVolumeSpecName "kube-api-access-x9fww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:29:00.482658 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.482615    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9fww\" (UniqueName: \"kubernetes.io/projected/2bf768c3-5645-44bc-82e4-9a4b04e87f13-kube-api-access-x9fww\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:00.482658 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.482650    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bf768c3-5645-44bc-82e4-9a4b04e87f13-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:00.538655 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.538517    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:00.658000 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.657955    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:00.705675 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.705635    2581 generic.go:356] "Generic (PLEG): container finished" podID="2bf768c3-5645-44bc-82e4-9a4b04e87f13" containerID="27ba31974cc83b1a59f67ed897d05970a7eb55b40098dd09c79f5cdb0e62bec7" exitCode=0
Sep 14 13:29:00.705675 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.705676    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"
Sep 14 13:29:00.705884 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.705711    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv" event={"ID":"2bf768c3-5645-44bc-82e4-9a4b04e87f13","Type":"ContainerDied","Data":"27ba31974cc83b1a59f67ed897d05970a7eb55b40098dd09c79f5cdb0e62bec7"}
Sep 14 13:29:00.705884 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.705785    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv" event={"ID":"2bf768c3-5645-44bc-82e4-9a4b04e87f13","Type":"ContainerDied","Data":"b6c5894d4fe417a45f1d555991c9b0e32768fca41c1500f4967ad9fa73ea3c6b"}
Sep 14 13:29:00.705884 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.705810    2581 scope.go:117] "RemoveContainer" containerID="27ba31974cc83b1a59f67ed897d05970a7eb55b40098dd09c79f5cdb0e62bec7"
Sep 14 13:29:00.715742 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.715724    2581 scope.go:117] "RemoveContainer" containerID="27ba31974cc83b1a59f67ed897d05970a7eb55b40098dd09c79f5cdb0e62bec7"
Sep 14 13:29:00.716058 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:29:00.716037    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27ba31974cc83b1a59f67ed897d05970a7eb55b40098dd09c79f5cdb0e62bec7\": container with ID starting with 27ba31974cc83b1a59f67ed897d05970a7eb55b40098dd09c79f5cdb0e62bec7 not found: ID does not exist" containerID="27ba31974cc83b1a59f67ed897d05970a7eb55b40098dd09c79f5cdb0e62bec7"
Sep 14 13:29:00.716118 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.716070    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27ba31974cc83b1a59f67ed897d05970a7eb55b40098dd09c79f5cdb0e62bec7"} err="failed to get container status \"27ba31974cc83b1a59f67ed897d05970a7eb55b40098dd09c79f5cdb0e62bec7\": rpc error: code = NotFound desc = could not find container \"27ba31974cc83b1a59f67ed897d05970a7eb55b40098dd09c79f5cdb0e62bec7\": container with ID starting with 27ba31974cc83b1a59f67ed897d05970a7eb55b40098dd09c79f5cdb0e62bec7 not found: ID does not exist"
Sep 14 13:29:00.727374 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.727342    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"]
Sep 14 13:29:00.733443 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:00.733416    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1cf25e-85f7554477-kvllv"]
Sep 14 13:29:01.246226 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:01.246188    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bf768c3-5645-44bc-82e4-9a4b04e87f13" path="/var/lib/kubelet/pods/2bf768c3-5645-44bc-82e4-9a4b04e87f13/volumes"
Sep 14 13:29:01.459220 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:01.459183    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:02.340302 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:02.340269    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:03.126992 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:03.126934    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:03.340835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:03.340786    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"]
Sep 14 13:29:03.358846 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:03.358814    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:03.658956 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:03.658920    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:29:03.820782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:03.820754    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:04.344597 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:04.344562    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:04.350493 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:04.350464    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"]
Sep 14 13:29:07.426425 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:07.426395    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:07.526578 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:07.526548    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:07.689892 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:07.689796    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:08.147469 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:08.147439    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:08.934672 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:08.934641    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:09.544834 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:09.544773    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:10.337200 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:10.337157    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:11.002996 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:11.002956    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:11.829808 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:11.829777    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:17.161413 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:17.161368    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:29:18.361026 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.360983    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8" podUID="509599fa-9990-4da1-ab73-9439d03a5f28" containerName="main" containerID="cri-o://e18498437619ca6fe11af310f6d9214e0a0994dd15fc4fe9493e40b2ccd65b95" gracePeriod=45
Sep 14 13:29:18.419431 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.419396    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:18.552116 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.552087    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:18.625234 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.625207    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:18.632176 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.632155    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:29:18.685874 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.685845    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:18.760868 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.760836    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:18.765924 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.765904    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-home\") pod \"509599fa-9990-4da1-ab73-9439d03a5f28\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") "
Sep 14 13:29:18.765996 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.765941    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-dshm\") pod \"509599fa-9990-4da1-ab73-9439d03a5f28\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") "
Sep 14 13:29:18.765996 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.765976    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-tmp-dir\") pod \"509599fa-9990-4da1-ab73-9439d03a5f28\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") "
Sep 14 13:29:18.766068 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.765997    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xsmx\" (UniqueName: \"kubernetes.io/projected/509599fa-9990-4da1-ab73-9439d03a5f28-kube-api-access-2xsmx\") pod \"509599fa-9990-4da1-ab73-9439d03a5f28\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") "
Sep 14 13:29:18.766068 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.766023    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-model-cache\") pod \"509599fa-9990-4da1-ab73-9439d03a5f28\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") "
Sep 14 13:29:18.766212 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.766190    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509599fa-9990-4da1-ab73-9439d03a5f28-tls-certs\") pod \"509599fa-9990-4da1-ab73-9439d03a5f28\" (UID: \"509599fa-9990-4da1-ab73-9439d03a5f28\") "
Sep 14 13:29:18.766271 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.766206    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-home" (OuterVolumeSpecName: "home") pod "509599fa-9990-4da1-ab73-9439d03a5f28" (UID: "509599fa-9990-4da1-ab73-9439d03a5f28"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:18.766329 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.766295    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-model-cache" (OuterVolumeSpecName: "model-cache") pod "509599fa-9990-4da1-ab73-9439d03a5f28" (UID: "509599fa-9990-4da1-ab73-9439d03a5f28"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:18.766329 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.766312    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "509599fa-9990-4da1-ab73-9439d03a5f28" (UID: "509599fa-9990-4da1-ab73-9439d03a5f28"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:18.766570 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.766522    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-tmp-dir\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:18.766570 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.766569    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-model-cache\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:18.766771 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.766581    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-home\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:18.768163 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.768136    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/509599fa-9990-4da1-ab73-9439d03a5f28-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "509599fa-9990-4da1-ab73-9439d03a5f28" (UID: "509599fa-9990-4da1-ab73-9439d03a5f28"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:29:18.768248 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.768172    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/509599fa-9990-4da1-ab73-9439d03a5f28-kube-api-access-2xsmx" (OuterVolumeSpecName: "kube-api-access-2xsmx") pod "509599fa-9990-4da1-ab73-9439d03a5f28" (UID: "509599fa-9990-4da1-ab73-9439d03a5f28"). InnerVolumeSpecName "kube-api-access-2xsmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:29:18.768248 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.768215    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-dshm" (OuterVolumeSpecName: "dshm") pod "509599fa-9990-4da1-ab73-9439d03a5f28" (UID: "509599fa-9990-4da1-ab73-9439d03a5f28"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:18.798972 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.798940    2581 generic.go:356] "Generic (PLEG): container finished" podID="509599fa-9990-4da1-ab73-9439d03a5f28" containerID="e18498437619ca6fe11af310f6d9214e0a0994dd15fc4fe9493e40b2ccd65b95" exitCode=0
Sep 14 13:29:18.799118 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.798982    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8" event={"ID":"509599fa-9990-4da1-ab73-9439d03a5f28","Type":"ContainerDied","Data":"e18498437619ca6fe11af310f6d9214e0a0994dd15fc4fe9493e40b2ccd65b95"}
Sep 14 13:29:18.799118 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.799004    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"
Sep 14 13:29:18.799118 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.799024    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8" event={"ID":"509599fa-9990-4da1-ab73-9439d03a5f28","Type":"ContainerDied","Data":"b00bed50ceb337ad17ae3d593679b140773948de5e910fe02e4034ddcf96841f"}
Sep 14 13:29:18.799118 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.799046    2581 scope.go:117] "RemoveContainer" containerID="e18498437619ca6fe11af310f6d9214e0a0994dd15fc4fe9493e40b2ccd65b95"
Sep 14 13:29:18.808621 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.808593    2581 scope.go:117] "RemoveContainer" containerID="e18498437619ca6fe11af310f6d9214e0a0994dd15fc4fe9493e40b2ccd65b95"
Sep 14 13:29:18.808919 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:29:18.808899    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e18498437619ca6fe11af310f6d9214e0a0994dd15fc4fe9493e40b2ccd65b95\": container with ID starting with e18498437619ca6fe11af310f6d9214e0a0994dd15fc4fe9493e40b2ccd65b95 not found: ID does not exist" containerID="e18498437619ca6fe11af310f6d9214e0a0994dd15fc4fe9493e40b2ccd65b95"
Sep 14 13:29:18.808999 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.808933    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e18498437619ca6fe11af310f6d9214e0a0994dd15fc4fe9493e40b2ccd65b95"} err="failed to get container status \"e18498437619ca6fe11af310f6d9214e0a0994dd15fc4fe9493e40b2ccd65b95\": rpc error: code = NotFound desc = could not find container \"e18498437619ca6fe11af310f6d9214e0a0994dd15fc4fe9493e40b2ccd65b95\": container with ID starting with e18498437619ca6fe11af310f6d9214e0a0994dd15fc4fe9493e40b2ccd65b95 not found: ID does not exist"
Sep 14 13:29:18.823082 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.823056    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"]
Sep 14 13:29:18.825032 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.825012    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14f3af/sub-test-model-14f3af-kserve-75b49459b9-ztzx8"]
Sep 14 13:29:18.867337 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.867276    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/509599fa-9990-4da1-ab73-9439d03a5f28-dshm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:18.867337 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.867298    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xsmx\" (UniqueName: \"kubernetes.io/projected/509599fa-9990-4da1-ab73-9439d03a5f28-kube-api-access-2xsmx\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:18.867337 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:18.867307    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/509599fa-9990-4da1-ab73-9439d03a5f28-tls-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:19.107868 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.107830    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:19.215272 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.215192    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:19.250088 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.250049    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="509599fa-9990-4da1-ab73-9439d03a5f28" path="/var/lib/kubelet/pods/509599fa-9990-4da1-ab73-9439d03a5f28/volumes"
Sep 14 13:29:19.370060 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.370018    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6" podUID="bd40f778-7b9c-44e3-b84a-d96fb071cfd2" containerName="main" containerID="cri-o://8f732d36465bbf053de845224532ac6f2c6855222f15888c3cd38f53522cd715" gracePeriod=45
Sep 14 13:29:19.624569 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.624543    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:29:19.775695 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.775604    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-dshm\") pod \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") "
Sep 14 13:29:19.775695 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.775640    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-model-cache\") pod \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") "
Sep 14 13:29:19.775949 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.775718    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-tmp-dir\") pod \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") "
Sep 14 13:29:19.775949 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.775749    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-tls-certs\") pod \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") "
Sep 14 13:29:19.775949 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.775797    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-home\") pod \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") "
Sep 14 13:29:19.775949 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.775842    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwrkm\" (UniqueName: \"kubernetes.io/projected/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-kube-api-access-vwrkm\") pod \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\" (UID: \"bd40f778-7b9c-44e3-b84a-d96fb071cfd2\") "
Sep 14 13:29:19.776154 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.775964    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bd40f778-7b9c-44e3-b84a-d96fb071cfd2" (UID: "bd40f778-7b9c-44e3-b84a-d96fb071cfd2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:19.776154 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.776002    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-model-cache" (OuterVolumeSpecName: "model-cache") pod "bd40f778-7b9c-44e3-b84a-d96fb071cfd2" (UID: "bd40f778-7b9c-44e3-b84a-d96fb071cfd2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:19.776257 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.776183    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-model-cache\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:19.776257 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.776203    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-tmp-dir\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:19.776339 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.776291    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-home" (OuterVolumeSpecName: "home") pod "bd40f778-7b9c-44e3-b84a-d96fb071cfd2" (UID: "bd40f778-7b9c-44e3-b84a-d96fb071cfd2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:19.777889 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.777862    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-dshm" (OuterVolumeSpecName: "dshm") pod "bd40f778-7b9c-44e3-b84a-d96fb071cfd2" (UID: "bd40f778-7b9c-44e3-b84a-d96fb071cfd2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:19.778004 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.777914    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-kube-api-access-vwrkm" (OuterVolumeSpecName: "kube-api-access-vwrkm") pod "bd40f778-7b9c-44e3-b84a-d96fb071cfd2" (UID: "bd40f778-7b9c-44e3-b84a-d96fb071cfd2"). InnerVolumeSpecName "kube-api-access-vwrkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:29:19.778004 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.777934    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bd40f778-7b9c-44e3-b84a-d96fb071cfd2" (UID: "bd40f778-7b9c-44e3-b84a-d96fb071cfd2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:29:19.807677 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.807639    2581 generic.go:356] "Generic (PLEG): container finished" podID="bd40f778-7b9c-44e3-b84a-d96fb071cfd2" containerID="8f732d36465bbf053de845224532ac6f2c6855222f15888c3cd38f53522cd715" exitCode=0
Sep 14 13:29:19.807882 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.807681    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"
Sep 14 13:29:19.807882 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.807732    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6" event={"ID":"bd40f778-7b9c-44e3-b84a-d96fb071cfd2","Type":"ContainerDied","Data":"8f732d36465bbf053de845224532ac6f2c6855222f15888c3cd38f53522cd715"}
Sep 14 13:29:19.807882 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.807777    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6" event={"ID":"bd40f778-7b9c-44e3-b84a-d96fb071cfd2","Type":"ContainerDied","Data":"69ca213778453e5352d7e2b18d2bf32ab73d8c2b1bcd0fca48b9c5ddcf6948c6"}
Sep 14 13:29:19.807882 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.807795    2581 scope.go:117] "RemoveContainer" containerID="8f732d36465bbf053de845224532ac6f2c6855222f15888c3cd38f53522cd715"
Sep 14 13:29:19.817606 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.817584    2581 scope.go:117] "RemoveContainer" containerID="8f732d36465bbf053de845224532ac6f2c6855222f15888c3cd38f53522cd715"
Sep 14 13:29:19.817939 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:29:19.817918    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f732d36465bbf053de845224532ac6f2c6855222f15888c3cd38f53522cd715\": container with ID starting with 8f732d36465bbf053de845224532ac6f2c6855222f15888c3cd38f53522cd715 not found: ID does not exist" containerID="8f732d36465bbf053de845224532ac6f2c6855222f15888c3cd38f53522cd715"
Sep 14 13:29:19.818014 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.817948    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f732d36465bbf053de845224532ac6f2c6855222f15888c3cd38f53522cd715"} err="failed to get container status \"8f732d36465bbf053de845224532ac6f2c6855222f15888c3cd38f53522cd715\": rpc error: code = NotFound desc = could not find container \"8f732d36465bbf053de845224532ac6f2c6855222f15888c3cd38f53522cd715\": container with ID starting with 8f732d36465bbf053de845224532ac6f2c6855222f15888c3cd38f53522cd715 not found: ID does not exist"
Sep 14 13:29:19.835724 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.835687    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"]
Sep 14 13:29:19.839307 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.839278    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/sub-test-model-eff5e8-kserve-79bb994c8f-9tld6"]
Sep 14 13:29:19.871089 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.871056    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:19.877304 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.877274    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-dshm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:19.877304 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.877301    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-tls-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:19.877304 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.877309    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-home\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:19.877522 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:19.877318    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwrkm\" (UniqueName: \"kubernetes.io/projected/bd40f778-7b9c-44e3-b84a-d96fb071cfd2-kube-api-access-vwrkm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:20.490128 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:20.490067    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:21.245449 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:21.245421    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:21.246952 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:21.246930    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd40f778-7b9c-44e3-b84a-d96fb071cfd2" path="/var/lib/kubelet/pods/bd40f778-7b9c-44e3-b84a-d96fb071cfd2/volumes"
Sep 14 13:29:21.642088 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:21.642051    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:25.346560 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:25.346511    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:26.580044 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:26.580008    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:26.683912 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:26.683879    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:27.416680 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:27.416651    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:28.057674 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:28.057638    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:28.761892 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:28.761854    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:29:28.792621 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:28.792589    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:29.535401 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:29.535367    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:30.963745 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:30.963711    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:36.477178 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:36.477141    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:37.112839 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.112800    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"]
Sep 14 13:29:37.113066 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.113038    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l" podUID="a91be069-cf4b-4719-b643-435475df1c26" containerName="maas-api" containerID="cri-o://36a01c20dbdc02155c27b9eda9aaaf176452935375bf3a476ce2a727653542fd" gracePeriod=30
Sep 14 13:29:37.143981 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.143942    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:37.207506 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.207465    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"]
Sep 14 13:29:37.207766 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.207738    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb" podUID="11b0249f-31cd-45d7-8540-cc3915a275a0" containerName="payload-processing" containerID="cri-o://7d908c65dc032d832e5692c0259107aea0be6d076f598b1aa499df76bef73aa3" gracePeriod=30
Sep 14 13:29:37.263043 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.263010    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:37.364155 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.364063    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:37.447635 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.447606    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:37.465949 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.465922    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:29:37.496818 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.496792    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"
Sep 14 13:29:37.532777 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.532739    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:37.551081 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.551048    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a91be069-cf4b-4719-b643-435475df1c26-openshift-service-ca\") pod \"a91be069-cf4b-4719-b643-435475df1c26\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") "
Sep 14 13:29:37.551257 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.551120    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5hcd\" (UniqueName: \"kubernetes.io/projected/11b0249f-31cd-45d7-8540-cc3915a275a0-kube-api-access-b5hcd\") pod \"11b0249f-31cd-45d7-8540-cc3915a275a0\" (UID: \"11b0249f-31cd-45d7-8540-cc3915a275a0\") "
Sep 14 13:29:37.551257 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.551145    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a91be069-cf4b-4719-b643-435475df1c26-maas-api-tls\") pod \"a91be069-cf4b-4719-b643-435475df1c26\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") "
Sep 14 13:29:37.551257 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.551170    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a91be069-cf4b-4719-b643-435475df1c26-metrics-certs\") pod \"a91be069-cf4b-4719-b643-435475df1c26\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") "
Sep 14 13:29:37.551257 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.551203    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtzl4\" (UniqueName: \"kubernetes.io/projected/a91be069-cf4b-4719-b643-435475df1c26-kube-api-access-xtzl4\") pod \"a91be069-cf4b-4719-b643-435475df1c26\" (UID: \"a91be069-cf4b-4719-b643-435475df1c26\") "
Sep 14 13:29:37.551257 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.551250    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11b0249f-31cd-45d7-8540-cc3915a275a0-plugins-config-volume\") pod \"11b0249f-31cd-45d7-8540-cc3915a275a0\" (UID: \"11b0249f-31cd-45d7-8540-cc3915a275a0\") "
Sep 14 13:29:37.551566 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.551494    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a91be069-cf4b-4719-b643-435475df1c26-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a91be069-cf4b-4719-b643-435475df1c26" (UID: "a91be069-cf4b-4719-b643-435475df1c26"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:29:37.551764 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.551732    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11b0249f-31cd-45d7-8540-cc3915a275a0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "11b0249f-31cd-45d7-8540-cc3915a275a0" (UID: "11b0249f-31cd-45d7-8540-cc3915a275a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:29:37.553565 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.553517    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a91be069-cf4b-4719-b643-435475df1c26-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a91be069-cf4b-4719-b643-435475df1c26" (UID: "a91be069-cf4b-4719-b643-435475df1c26"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:29:37.553677 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.553561    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a91be069-cf4b-4719-b643-435475df1c26-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a91be069-cf4b-4719-b643-435475df1c26" (UID: "a91be069-cf4b-4719-b643-435475df1c26"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:29:37.553677 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.553589    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a91be069-cf4b-4719-b643-435475df1c26-kube-api-access-xtzl4" (OuterVolumeSpecName: "kube-api-access-xtzl4") pod "a91be069-cf4b-4719-b643-435475df1c26" (UID: "a91be069-cf4b-4719-b643-435475df1c26"). InnerVolumeSpecName "kube-api-access-xtzl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:29:37.553677 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.553613    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11b0249f-31cd-45d7-8540-cc3915a275a0-kube-api-access-b5hcd" (OuterVolumeSpecName: "kube-api-access-b5hcd") pod "11b0249f-31cd-45d7-8540-cc3915a275a0" (UID: "11b0249f-31cd-45d7-8540-cc3915a275a0"). InnerVolumeSpecName "kube-api-access-b5hcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:29:37.652503 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.652417    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11b0249f-31cd-45d7-8540-cc3915a275a0-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:37.652503 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.652447    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a91be069-cf4b-4719-b643-435475df1c26-openshift-service-ca\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:37.652503 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.652462    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5hcd\" (UniqueName: \"kubernetes.io/projected/11b0249f-31cd-45d7-8540-cc3915a275a0-kube-api-access-b5hcd\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:37.652503 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.652472    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a91be069-cf4b-4719-b643-435475df1c26-maas-api-tls\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:37.652503 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.652480    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a91be069-cf4b-4719-b643-435475df1c26-metrics-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:37.652503 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.652489    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtzl4\" (UniqueName: \"kubernetes.io/projected/a91be069-cf4b-4719-b643-435475df1c26-kube-api-access-xtzl4\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:29:37.842647 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.842608    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:37.898482 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.898442    2581 generic.go:356] "Generic (PLEG): container finished" podID="11b0249f-31cd-45d7-8540-cc3915a275a0" containerID="7d908c65dc032d832e5692c0259107aea0be6d076f598b1aa499df76bef73aa3" exitCode=0
Sep 14 13:29:37.898681 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.898493    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"
Sep 14 13:29:37.898681 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.898562    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb" event={"ID":"11b0249f-31cd-45d7-8540-cc3915a275a0","Type":"ContainerDied","Data":"7d908c65dc032d832e5692c0259107aea0be6d076f598b1aa499df76bef73aa3"}
Sep 14 13:29:37.898681 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.898629    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb" event={"ID":"11b0249f-31cd-45d7-8540-cc3915a275a0","Type":"ContainerDied","Data":"3d4b9b096b2711ebde2a95a4c08324c6196757a739096f2fc7b7176903a81c9a"}
Sep 14 13:29:37.898681 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.898660    2581 scope.go:117] "RemoveContainer" containerID="7d908c65dc032d832e5692c0259107aea0be6d076f598b1aa499df76bef73aa3"
Sep 14 13:29:37.900065 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.900043    2581 generic.go:356] "Generic (PLEG): container finished" podID="a91be069-cf4b-4719-b643-435475df1c26" containerID="36a01c20dbdc02155c27b9eda9aaaf176452935375bf3a476ce2a727653542fd" exitCode=0
Sep 14 13:29:37.900193 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.900093    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"
Sep 14 13:29:37.900193 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.900119    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l" event={"ID":"a91be069-cf4b-4719-b643-435475df1c26","Type":"ContainerDied","Data":"36a01c20dbdc02155c27b9eda9aaaf176452935375bf3a476ce2a727653542fd"}
Sep 14 13:29:37.900193 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.900161    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l" event={"ID":"a91be069-cf4b-4719-b643-435475df1c26","Type":"ContainerDied","Data":"076f4dcb7b2923eb40a73ccab856fe521e7c987810d7e7f86a3e9d788e0008d5"}
Sep 14 13:29:37.909214 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.909185    2581 scope.go:117] "RemoveContainer" containerID="7d908c65dc032d832e5692c0259107aea0be6d076f598b1aa499df76bef73aa3"
Sep 14 13:29:37.909555 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:29:37.909514    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d908c65dc032d832e5692c0259107aea0be6d076f598b1aa499df76bef73aa3\": container with ID starting with 7d908c65dc032d832e5692c0259107aea0be6d076f598b1aa499df76bef73aa3 not found: ID does not exist" containerID="7d908c65dc032d832e5692c0259107aea0be6d076f598b1aa499df76bef73aa3"
Sep 14 13:29:37.909634 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.909558    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d908c65dc032d832e5692c0259107aea0be6d076f598b1aa499df76bef73aa3"} err="failed to get container status \"7d908c65dc032d832e5692c0259107aea0be6d076f598b1aa499df76bef73aa3\": rpc error: code = NotFound desc = could not find container \"7d908c65dc032d832e5692c0259107aea0be6d076f598b1aa499df76bef73aa3\": container with ID starting with 7d908c65dc032d832e5692c0259107aea0be6d076f598b1aa499df76bef73aa3 not found: ID does not exist"
Sep 14 13:29:37.909634 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.909581    2581 scope.go:117] "RemoveContainer" containerID="36a01c20dbdc02155c27b9eda9aaaf176452935375bf3a476ce2a727653542fd"
Sep 14 13:29:37.918493 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.918473    2581 scope.go:117] "RemoveContainer" containerID="36a01c20dbdc02155c27b9eda9aaaf176452935375bf3a476ce2a727653542fd"
Sep 14 13:29:37.918783 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:29:37.918764    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36a01c20dbdc02155c27b9eda9aaaf176452935375bf3a476ce2a727653542fd\": container with ID starting with 36a01c20dbdc02155c27b9eda9aaaf176452935375bf3a476ce2a727653542fd not found: ID does not exist" containerID="36a01c20dbdc02155c27b9eda9aaaf176452935375bf3a476ce2a727653542fd"
Sep 14 13:29:37.918852 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.918791    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36a01c20dbdc02155c27b9eda9aaaf176452935375bf3a476ce2a727653542fd"} err="failed to get container status \"36a01c20dbdc02155c27b9eda9aaaf176452935375bf3a476ce2a727653542fd\": rpc error: code = NotFound desc = could not find container \"36a01c20dbdc02155c27b9eda9aaaf176452935375bf3a476ce2a727653542fd\": container with ID starting with 36a01c20dbdc02155c27b9eda9aaaf176452935375bf3a476ce2a727653542fd not found: ID does not exist"
Sep 14 13:29:37.924398 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.924370    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"]
Sep 14 13:29:37.934835 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.934801    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b8be88-5d65b99d5c-vs8pb"]
Sep 14 13:29:37.948506 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.948455    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"]
Sep 14 13:29:37.950826 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.950798    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-b8be88-79b7c756b8-5p86l"]
Sep 14 13:29:37.973349 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:37.973306    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:38.737553 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:38.737505    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:39.245976 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:39.245943    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11b0249f-31cd-45d7-8540-cc3915a275a0" path="/var/lib/kubelet/pods/11b0249f-31cd-45d7-8540-cc3915a275a0/volumes"
Sep 14 13:29:39.246373 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:39.246359    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a91be069-cf4b-4719-b643-435475df1c26" path="/var/lib/kubelet/pods/a91be069-cf4b-4719-b643-435475df1c26/volumes"
Sep 14 13:29:39.432937 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:39.432906    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:42.372489 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:42.372454    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:29:44.900999 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:44.900961    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:44.968661 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:44.968630    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:45.150407 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.150374    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:45.792272 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792225    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"]
Sep 14 13:29:45.792623 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792608    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd40f778-7b9c-44e3-b84a-d96fb071cfd2" containerName="main"
Sep 14 13:29:45.792623 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792622    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd40f778-7b9c-44e3-b84a-d96fb071cfd2" containerName="main"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792632    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a91be069-cf4b-4719-b643-435475df1c26" containerName="maas-api"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792637    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a91be069-cf4b-4719-b643-435475df1c26" containerName="maas-api"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792651    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40d1e5b8-8a69-4090-b2ba-87f00d11dd8c" containerName="maas-api"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792657    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="40d1e5b8-8a69-4090-b2ba-87f00d11dd8c" containerName="maas-api"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792667    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e673c82c-e41b-417a-bf11-0a9b3caef716" containerName="payload-pre-processing"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792674    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e673c82c-e41b-417a-bf11-0a9b3caef716" containerName="payload-pre-processing"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792682    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bf768c3-5645-44bc-82e4-9a4b04e87f13" containerName="payload-processing"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792688    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bf768c3-5645-44bc-82e4-9a4b04e87f13" containerName="payload-processing"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792696    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11b0249f-31cd-45d7-8540-cc3915a275a0" containerName="payload-processing"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792700    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="11b0249f-31cd-45d7-8540-cc3915a275a0" containerName="payload-processing"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792710    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="509599fa-9990-4da1-ab73-9439d03a5f28" containerName="main"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792715    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="509599fa-9990-4da1-ab73-9439d03a5f28" containerName="main"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792765    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="11b0249f-31cd-45d7-8540-cc3915a275a0" containerName="payload-processing"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792772    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a91be069-cf4b-4719-b643-435475df1c26" containerName="maas-api"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792778    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e673c82c-e41b-417a-bf11-0a9b3caef716" containerName="payload-pre-processing"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792785    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="509599fa-9990-4da1-ab73-9439d03a5f28" containerName="main"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792791    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="40d1e5b8-8a69-4090-b2ba-87f00d11dd8c" containerName="maas-api"
Sep 14 13:29:45.792782 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792796    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bf768c3-5645-44bc-82e4-9a4b04e87f13" containerName="payload-processing"
Sep 14 13:29:45.793842 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.792803    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd40f778-7b9c-44e3-b84a-d96fb071cfd2" containerName="main"
Sep 14 13:29:45.797458 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.797435    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:45.801095 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.801067    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-eff5e8\"/\"kube-root-ca.crt\""
Sep 14 13:29:45.801211 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.801162    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-eff5e8\"/\"rate-test-model-eff5e8-kserve-self-signed-certs\""
Sep 14 13:29:45.801211 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.801182    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-eff5e8\"/\"default-dockercfg-hhsnb\""
Sep 14 13:29:45.801329 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.801184    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-eff5e8\"/\"openshift-service-ca.crt\""
Sep 14 13:29:45.808637 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.808614    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"]
Sep 14 13:29:45.813508 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.813480    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:45.927234 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.927172    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-dshm\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:45.927234 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.927238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-tmp-dir\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:45.927908 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.927350    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16118cd1-74c4-4c15-8528-13c39ee87a47-tls-certs\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:45.927908 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.927414    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-model-cache\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:45.927908 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.927569    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvxq2\" (UniqueName: \"kubernetes.io/projected/16118cd1-74c4-4c15-8528-13c39ee87a47-kube-api-access-gvxq2\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:45.927908 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:45.927627    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-home\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:46.029012 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.028970    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvxq2\" (UniqueName: \"kubernetes.io/projected/16118cd1-74c4-4c15-8528-13c39ee87a47-kube-api-access-gvxq2\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:46.029239 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.029032    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-home\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:46.029239 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.029078    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-dshm\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:46.029239 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.029105    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-tmp-dir\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:46.029239 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.029134    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16118cd1-74c4-4c15-8528-13c39ee87a47-tls-certs\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:46.029239 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.029172    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-model-cache\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:46.029565 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.029524    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-home\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:46.029647 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.029590    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-tmp-dir\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:46.029647 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.029617    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-model-cache\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:46.031638 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.031614    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-dshm\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:46.031793 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.031772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16118cd1-74c4-4c15-8528-13c39ee87a47-tls-certs\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:46.037743 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.037722    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvxq2\" (UniqueName: \"kubernetes.io/projected/16118cd1-74c4-4c15-8528-13c39ee87a47-kube-api-access-gvxq2\") pod \"rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") " pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:46.107735 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.107685    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:46.394057 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.394028    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"]
Sep 14 13:29:46.396663 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:29:46.396640    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16118cd1_74c4_4c15_8528_13c39ee87a47.slice/crio-bc949f270681c23ce6bc2c2d69ca4e7bbc0637702f26176a95d6a6444b79afcc WatchSource:0}: Error finding container bc949f270681c23ce6bc2c2d69ca4e7bbc0637702f26176a95d6a6444b79afcc: Status 404 returned error can't find the container with id bc949f270681c23ce6bc2c2d69ca4e7bbc0637702f26176a95d6a6444b79afcc
Sep 14 13:29:46.951616 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.951575    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh" event={"ID":"16118cd1-74c4-4c15-8528-13c39ee87a47","Type":"ContainerStarted","Data":"17d573d7b23e9078360d56f88a8f1cb2108df156e6668a90c49431e6f4a1c57a"}
Sep 14 13:29:46.952050 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.951625    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh" event={"ID":"16118cd1-74c4-4c15-8528-13c39ee87a47","Type":"ContainerStarted","Data":"bc949f270681c23ce6bc2c2d69ca4e7bbc0637702f26176a95d6a6444b79afcc"}
Sep 14 13:29:46.970294 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:46.970247    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh" podStartSLOduration=1.970231694 podStartE2EDuration="1.970231694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:29:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:29:46.969044607 +0000 UTC m=+1880.260378425" watchObservedRunningTime="2026-09-14 13:29:46.970231694 +0000 UTC m=+1880.261565549"
Sep 14 13:29:55.562708 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:55.562625    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:29:56.108645 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:56.108608    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:56.108829 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:56.108659    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:56.121172 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:56.121146    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:57.015212 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:57.015185    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:29:58.493209 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:29:58.493174    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:00.145410 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:00.145371    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-eff5e8-29823210-44x58"]
Sep 14 13:30:00.149476 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:00.149453    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-eff5e8-29823210-44x58"
Sep 14 13:30:00.152035 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:00.152009    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vbbzp\""
Sep 14 13:30:00.157450 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:00.157425    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-eff5e8-29823210-44x58"]
Sep 14 13:30:00.164062 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:00.164043    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-eff5e8-29823210-44x58"
Sep 14 13:30:00.435873 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:00.435845    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-eff5e8-29823210-44x58"]
Sep 14 13:30:00.438448 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:30:00.438423    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb70d3dba_151d_4e33_8da7_08b51bf64559.slice/crio-072522975f814910f25d5565359e6ec6a4b63bd3aae1bbd333660a6eb18c5b4b WatchSource:0}: Error finding container 072522975f814910f25d5565359e6ec6a4b63bd3aae1bbd333660a6eb18c5b4b: Status 404 returned error can't find the container with id 072522975f814910f25d5565359e6ec6a4b63bd3aae1bbd333660a6eb18c5b4b
Sep 14 13:30:01.032647 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:01.032524    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-eff5e8-29823210-44x58" event={"ID":"b70d3dba-151d-4e33-8da7-08b51bf64559","Type":"ContainerStarted","Data":"13bb883b4993aea0b92df592876629b3c533b74b59a4879241d131504d77a0b0"}
Sep 14 13:30:01.032796 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:01.032647    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-eff5e8-29823210-44x58" event={"ID":"b70d3dba-151d-4e33-8da7-08b51bf64559","Type":"ContainerStarted","Data":"072522975f814910f25d5565359e6ec6a4b63bd3aae1bbd333660a6eb18c5b4b"}
Sep 14 13:30:01.049248 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:01.049188    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-eff5e8-29823210-44x58" podStartSLOduration=1.049173113 podStartE2EDuration="1.049173113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:30:01.048972231 +0000 UTC m=+1894.340306048" watchObservedRunningTime="2026-09-14 13:30:01.049173113 +0000 UTC m=+1894.340506963"
Sep 14 13:30:02.038949 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:02.038908    2581 generic.go:356] "Generic (PLEG): container finished" podID="b70d3dba-151d-4e33-8da7-08b51bf64559" containerID="13bb883b4993aea0b92df592876629b3c533b74b59a4879241d131504d77a0b0" exitCode=0
Sep 14 13:30:02.039386 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:02.038999    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-eff5e8-29823210-44x58" event={"ID":"b70d3dba-151d-4e33-8da7-08b51bf64559","Type":"ContainerDied","Data":"13bb883b4993aea0b92df592876629b3c533b74b59a4879241d131504d77a0b0"}
Sep 14 13:30:03.184621 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:03.184592    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-eff5e8-29823210-44x58"
Sep 14 13:30:04.048198 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:04.048160    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:30:04.050766 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:04.050744    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-eff5e8-29823210-44x58"
Sep 14 13:30:04.050892 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:04.050782    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-eff5e8-29823210-44x58" event={"ID":"b70d3dba-151d-4e33-8da7-08b51bf64559","Type":"ContainerDied","Data":"072522975f814910f25d5565359e6ec6a4b63bd3aae1bbd333660a6eb18c5b4b"}
Sep 14 13:30:04.050892 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:04.050827    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="072522975f814910f25d5565359e6ec6a4b63bd3aae1bbd333660a6eb18c5b4b"
Sep 14 13:30:13.475014 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:13.474973    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:30:18.559112 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:18.559079    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:19.496841 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:19.496806    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"]
Sep 14 13:30:19.520318 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:19.520287    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:19.968810 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:19.968776    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:20.271890 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:20.271804    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:20.405802 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:20.405767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:22.475971 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:22.475937    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:23.129652 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:23.129605    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:23.225145 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:23.225112    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:24.160834 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:24.160782    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:24.902582 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:24.902548    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:24.968241 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:24.968211    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:25.100053 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.100023    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:25.402237 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.402205    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:25.503020 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.502991    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:25.671623 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.671547    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"]
Sep 14 13:30:25.671950 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.671938    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b70d3dba-151d-4e33-8da7-08b51bf64559" containerName="cleanup"
Sep 14 13:30:25.671995 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.671952    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b70d3dba-151d-4e33-8da7-08b51bf64559" containerName="cleanup"
Sep 14 13:30:25.672046 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.672037    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b70d3dba-151d-4e33-8da7-08b51bf64559" containerName="cleanup"
Sep 14 13:30:25.675281 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.675265    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"
Sep 14 13:30:25.678265 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.678237    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-c87a95\""
Sep 14 13:30:25.678397 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.678334    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-c87a95-dockercfg-q8494\""
Sep 14 13:30:25.685607 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.685582    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"]
Sep 14 13:30:25.808992 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.808956    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5860a73f-c28e-4c62-8611-cc126fae5cae-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8\" (UID: \"5860a73f-c28e-4c62-8611-cc126fae5cae\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"
Sep 14 13:30:25.808992 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.809000    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl7hs\" (UniqueName: \"kubernetes.io/projected/5860a73f-c28e-4c62-8611-cc126fae5cae-kube-api-access-tl7hs\") pod \"payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8\" (UID: \"5860a73f-c28e-4c62-8611-cc126fae5cae\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"
Sep 14 13:30:25.910439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.910405    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5860a73f-c28e-4c62-8611-cc126fae5cae-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8\" (UID: \"5860a73f-c28e-4c62-8611-cc126fae5cae\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"
Sep 14 13:30:25.910439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.910442    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl7hs\" (UniqueName: \"kubernetes.io/projected/5860a73f-c28e-4c62-8611-cc126fae5cae-kube-api-access-tl7hs\") pod \"payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8\" (UID: \"5860a73f-c28e-4c62-8611-cc126fae5cae\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"
Sep 14 13:30:25.911116 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.911096    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5860a73f-c28e-4c62-8611-cc126fae5cae-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8\" (UID: \"5860a73f-c28e-4c62-8611-cc126fae5cae\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"
Sep 14 13:30:25.919017 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.918999    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl7hs\" (UniqueName: \"kubernetes.io/projected/5860a73f-c28e-4c62-8611-cc126fae5cae-kube-api-access-tl7hs\") pod \"payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8\" (UID: \"5860a73f-c28e-4c62-8611-cc126fae5cae\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"
Sep 14 13:30:25.986109 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:25.986015    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"
Sep 14 13:30:26.265720 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:26.265696    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"]
Sep 14 13:30:26.267447 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:30:26.267419    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5860a73f_c28e_4c62_8611_cc126fae5cae.slice/crio-6ca0f3a852dbb3ee631f9678d7db11a0365164a9de6a1d22b4d26d23d9488858 WatchSource:0}: Error finding container 6ca0f3a852dbb3ee631f9678d7db11a0365164a9de6a1d22b4d26d23d9488858: Status 404 returned error can't find the container with id 6ca0f3a852dbb3ee631f9678d7db11a0365164a9de6a1d22b4d26d23d9488858
Sep 14 13:30:26.465784 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:26.465749    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:30:27.167889 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:27.167848    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8" event={"ID":"5860a73f-c28e-4c62-8611-cc126fae5cae","Type":"ContainerStarted","Data":"68bbac89327e4f16a51e075ba4f410c7855d17088f77f3a0a9d82eb82088c120"}
Sep 14 13:30:27.168104 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:27.167908    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8" event={"ID":"5860a73f-c28e-4c62-8611-cc126fae5cae","Type":"ContainerStarted","Data":"6ca0f3a852dbb3ee631f9678d7db11a0365164a9de6a1d22b4d26d23d9488858"}
Sep 14 13:30:27.168104 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:27.167934    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"
Sep 14 13:30:27.190744 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:27.190689    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8" podStartSLOduration=2.190668868 podStartE2EDuration="2.190668868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:30:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:30:27.188422024 +0000 UTC m=+1920.479755842" watchObservedRunningTime="2026-09-14 13:30:27.190668868 +0000 UTC m=+1920.482002725"
Sep 14 13:30:34.515907 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.515866    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh" podUID="16118cd1-74c4-4c15-8528-13c39ee87a47" containerName="main" containerID="cri-o://17d573d7b23e9078360d56f88a8f1cb2108df156e6668a90c49431e6f4a1c57a" gracePeriod=45
Sep 14 13:30:34.768189 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.768128    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:30:34.895432 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.895400    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-home\") pod \"16118cd1-74c4-4c15-8528-13c39ee87a47\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") "
Sep 14 13:30:34.895631 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.895461    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16118cd1-74c4-4c15-8528-13c39ee87a47-tls-certs\") pod \"16118cd1-74c4-4c15-8528-13c39ee87a47\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") "
Sep 14 13:30:34.895631 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.895584    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-tmp-dir\") pod \"16118cd1-74c4-4c15-8528-13c39ee87a47\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") "
Sep 14 13:30:34.895631 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.895613    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-model-cache\") pod \"16118cd1-74c4-4c15-8528-13c39ee87a47\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") "
Sep 14 13:30:34.895805 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.895639    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-dshm\") pod \"16118cd1-74c4-4c15-8528-13c39ee87a47\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") "
Sep 14 13:30:34.895805 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.895646    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-home" (OuterVolumeSpecName: "home") pod "16118cd1-74c4-4c15-8528-13c39ee87a47" (UID: "16118cd1-74c4-4c15-8528-13c39ee87a47"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:30:34.895805 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.895664    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvxq2\" (UniqueName: \"kubernetes.io/projected/16118cd1-74c4-4c15-8528-13c39ee87a47-kube-api-access-gvxq2\") pod \"16118cd1-74c4-4c15-8528-13c39ee87a47\" (UID: \"16118cd1-74c4-4c15-8528-13c39ee87a47\") "
Sep 14 13:30:34.895944 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.895859    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-model-cache" (OuterVolumeSpecName: "model-cache") pod "16118cd1-74c4-4c15-8528-13c39ee87a47" (UID: "16118cd1-74c4-4c15-8528-13c39ee87a47"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:30:34.895944 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.895870    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "16118cd1-74c4-4c15-8528-13c39ee87a47" (UID: "16118cd1-74c4-4c15-8528-13c39ee87a47"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:30:34.896016 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.895980    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-tmp-dir\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:30:34.896016 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.895997    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-model-cache\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:30:34.896016 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.896011    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-home\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:30:34.897617 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.897590    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-dshm" (OuterVolumeSpecName: "dshm") pod "16118cd1-74c4-4c15-8528-13c39ee87a47" (UID: "16118cd1-74c4-4c15-8528-13c39ee87a47"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:30:34.897617 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.897589    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16118cd1-74c4-4c15-8528-13c39ee87a47-kube-api-access-gvxq2" (OuterVolumeSpecName: "kube-api-access-gvxq2") pod "16118cd1-74c4-4c15-8528-13c39ee87a47" (UID: "16118cd1-74c4-4c15-8528-13c39ee87a47"). InnerVolumeSpecName "kube-api-access-gvxq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:30:34.897757 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.897690    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16118cd1-74c4-4c15-8528-13c39ee87a47-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "16118cd1-74c4-4c15-8528-13c39ee87a47" (UID: "16118cd1-74c4-4c15-8528-13c39ee87a47"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:30:34.959825 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.959800    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:30:34.997108 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.997089    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvxq2\" (UniqueName: \"kubernetes.io/projected/16118cd1-74c4-4c15-8528-13c39ee87a47-kube-api-access-gvxq2\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:30:34.997108 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.997108    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16118cd1-74c4-4c15-8528-13c39ee87a47-tls-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:30:34.997257 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:34.997119    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16118cd1-74c4-4c15-8528-13c39ee87a47-dshm\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:30:35.208919 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:35.208876    2581 generic.go:356] "Generic (PLEG): container finished" podID="16118cd1-74c4-4c15-8528-13c39ee87a47" containerID="17d573d7b23e9078360d56f88a8f1cb2108df156e6668a90c49431e6f4a1c57a" exitCode=0
Sep 14 13:30:35.208919 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:35.208920    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"
Sep 14 13:30:35.209198 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:35.208958    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh" event={"ID":"16118cd1-74c4-4c15-8528-13c39ee87a47","Type":"ContainerDied","Data":"17d573d7b23e9078360d56f88a8f1cb2108df156e6668a90c49431e6f4a1c57a"}
Sep 14 13:30:35.209198 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:35.209009    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh" event={"ID":"16118cd1-74c4-4c15-8528-13c39ee87a47","Type":"ContainerDied","Data":"bc949f270681c23ce6bc2c2d69ca4e7bbc0637702f26176a95d6a6444b79afcc"}
Sep 14 13:30:35.209198 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:35.209034    2581 scope.go:117] "RemoveContainer" containerID="17d573d7b23e9078360d56f88a8f1cb2108df156e6668a90c49431e6f4a1c57a"
Sep 14 13:30:35.217997 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:35.217981    2581 scope.go:117] "RemoveContainer" containerID="17d573d7b23e9078360d56f88a8f1cb2108df156e6668a90c49431e6f4a1c57a"
Sep 14 13:30:35.218262 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:30:35.218246    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17d573d7b23e9078360d56f88a8f1cb2108df156e6668a90c49431e6f4a1c57a\": container with ID starting with 17d573d7b23e9078360d56f88a8f1cb2108df156e6668a90c49431e6f4a1c57a not found: ID does not exist" containerID="17d573d7b23e9078360d56f88a8f1cb2108df156e6668a90c49431e6f4a1c57a"
Sep 14 13:30:35.218312 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:35.218271    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17d573d7b23e9078360d56f88a8f1cb2108df156e6668a90c49431e6f4a1c57a"} err="failed to get container status \"17d573d7b23e9078360d56f88a8f1cb2108df156e6668a90c49431e6f4a1c57a\": rpc error: code = NotFound desc = could not find container \"17d573d7b23e9078360d56f88a8f1cb2108df156e6668a90c49431e6f4a1c57a\": container with ID starting with 17d573d7b23e9078360d56f88a8f1cb2108df156e6668a90c49431e6f4a1c57a not found: ID does not exist"
Sep 14 13:30:35.230753 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:35.230731    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"]
Sep 14 13:30:35.234864 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:35.234846    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-eff5e8/rate-test-model-eff5e8-kserve-5bc9995c87-dpvxh"]
Sep 14 13:30:35.246003 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:35.245980    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16118cd1-74c4-4c15-8528-13c39ee87a47" path="/var/lib/kubelet/pods/16118cd1-74c4-4c15-8528-13c39ee87a47/volumes"
Sep 14 13:30:38.176193 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:38.176165    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"
Sep 14 13:30:47.148007 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:47.147966    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:47.837942 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:47.837909    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:48.092592 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:48.092503    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:48.714330 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:48.714296    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:48.813860 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:48.813832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:49.081117 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:49.081032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:49.349968 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:49.349891    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:49.499112 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:49.499082    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:51.693000 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:51.692964    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:52.329346 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:52.329317    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:52.436403 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:52.436365    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:53.236339 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:53.236310    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:53.908859 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:53.908832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:54.584692 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:54.584665    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:54.685163 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:54.685131    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:55.082298 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.082259    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:55.185754 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.185725    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:55.284043 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.284012    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"]
Sep 14 13:30:55.284487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.284468    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16118cd1-74c4-4c15-8528-13c39ee87a47" containerName="main"
Sep 14 13:30:55.284487 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.284486    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="16118cd1-74c4-4c15-8528-13c39ee87a47" containerName="main"
Sep 14 13:30:55.284651 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.284577    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="16118cd1-74c4-4c15-8528-13c39ee87a47" containerName="main"
Sep 14 13:30:55.289106 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.289088    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"
Sep 14 13:30:55.291706 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.291663    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-27ae76\""
Sep 14 13:30:55.291818 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.291734    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-27ae76-dockercfg-f8p7s\""
Sep 14 13:30:55.294426 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.294397    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:55.299123 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.299103    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"]
Sep 14 13:30:55.384983 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.384953    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21c13b16-51cf-4a74-81f0-2e9e4296dfa0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr\" (UID: \"21c13b16-51cf-4a74-81f0-2e9e4296dfa0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"
Sep 14 13:30:55.385142 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.385016    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brv6n\" (UniqueName: \"kubernetes.io/projected/21c13b16-51cf-4a74-81f0-2e9e4296dfa0-kube-api-access-brv6n\") pod \"payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr\" (UID: \"21c13b16-51cf-4a74-81f0-2e9e4296dfa0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"
Sep 14 13:30:55.485774 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.485741    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21c13b16-51cf-4a74-81f0-2e9e4296dfa0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr\" (UID: \"21c13b16-51cf-4a74-81f0-2e9e4296dfa0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"
Sep 14 13:30:55.485936 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.485794    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brv6n\" (UniqueName: \"kubernetes.io/projected/21c13b16-51cf-4a74-81f0-2e9e4296dfa0-kube-api-access-brv6n\") pod \"payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr\" (UID: \"21c13b16-51cf-4a74-81f0-2e9e4296dfa0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"
Sep 14 13:30:55.486463 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.486443    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21c13b16-51cf-4a74-81f0-2e9e4296dfa0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr\" (UID: \"21c13b16-51cf-4a74-81f0-2e9e4296dfa0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"
Sep 14 13:30:55.493667 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.493646    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brv6n\" (UniqueName: \"kubernetes.io/projected/21c13b16-51cf-4a74-81f0-2e9e4296dfa0-kube-api-access-brv6n\") pod \"payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr\" (UID: \"21c13b16-51cf-4a74-81f0-2e9e4296dfa0\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"
Sep 14 13:30:55.601647 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.601603    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"
Sep 14 13:30:55.872769 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:55.872739    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"]
Sep 14 13:30:55.875412 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:30:55.875385    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21c13b16_51cf_4a74_81f0_2e9e4296dfa0.slice/crio-94cc24326c86dd822d74a75921e40689f05d0d92db34854c3d75d4bbbf9dfa4a WatchSource:0}: Error finding container 94cc24326c86dd822d74a75921e40689f05d0d92db34854c3d75d4bbbf9dfa4a: Status 404 returned error can't find the container with id 94cc24326c86dd822d74a75921e40689f05d0d92db34854c3d75d4bbbf9dfa4a
Sep 14 13:30:56.325110 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:56.325005    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr" event={"ID":"21c13b16-51cf-4a74-81f0-2e9e4296dfa0","Type":"ContainerStarted","Data":"a2c8b93fc3c607527932de1629d1484687dde7b954495b051d3b5db15d5977fb"}
Sep 14 13:30:56.325110 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:56.325067    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr" event={"ID":"21c13b16-51cf-4a74-81f0-2e9e4296dfa0","Type":"ContainerStarted","Data":"94cc24326c86dd822d74a75921e40689f05d0d92db34854c3d75d4bbbf9dfa4a"}
Sep 14 13:30:56.325110 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:56.325092    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"
Sep 14 13:30:56.345552 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:56.345493    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr" podStartSLOduration=1.345479128 podStartE2EDuration="1.345479128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:30:56.342717582 +0000 UTC m=+1949.634051399" watchObservedRunningTime="2026-09-14 13:30:56.345479128 +0000 UTC m=+1949.636812948"
Sep 14 13:30:59.963912 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:30:59.963879    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:31:07.332689 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:07.332657    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"
Sep 14 13:31:16.744364 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:16.744332    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:31:17.301174 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:17.301136    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:31:17.429886 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:17.429832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:31:18.103323 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:18.103277    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:31:18.856101 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:18.856053    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:31:19.509159 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:19.509125    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:31:22.370522 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:22.370493    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr_21c13b16-51cf-4a74-81f0-2e9e4296dfa0/payload-processing/0.log"
Sep 14 13:31:31.735328 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:31.735273    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:31:34.688259 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:34.688222    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:31:34.789202 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:34.789162    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:31:35.559645 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:35.559614    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:31:51.855925 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:51.855891    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:31:58.109697 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:58.109661    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8_5860a73f-c28e-4c62-8611-cc126fae5cae/payload-processing/0.log"
Sep 14 13:31:58.340368 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:58.340343    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr_21c13b16-51cf-4a74-81f0-2e9e4296dfa0/payload-processing/0.log"
Sep 14 13:31:58.597749 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:58.597674    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:31:59.153777 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:59.153742    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:31:59.248885 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:31:59.248860    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:00.836249 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:00.836204    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:01.650256 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:01.650219    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:01.778261 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:01.778220    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:02.358963 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:02.358915    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:03.297265 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:03.297237    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:04.035682 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.035645    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:04.722452 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.722420    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"]
Sep 14 13:32:04.726363 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.726344    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:04.728916 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.728898    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-8e2f31-serving-cert\""
Sep 14 13:32:04.741398 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.741375    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"]
Sep 14 13:32:04.796426 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.796397    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:04.796590 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.796427    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xplh7\" (UniqueName: \"kubernetes.io/projected/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-kube-api-access-xplh7\") pod \"maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:04.796590 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.796456    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:04.796590 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.796479    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:04.810330 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.810300    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:04.897488 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.897458    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:04.897488 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.897490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xplh7\" (UniqueName: \"kubernetes.io/projected/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-kube-api-access-xplh7\") pod \"maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:04.897689 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.897517    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:04.897689 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.897600    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:04.897689 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:32:04.897633    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-serving-cert: secret "maas-api-e2e-ipp-cleanup-8e2f31-serving-cert" not found
Sep 14 13:32:04.897689 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:32:04.897686    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-maas-api-tls podName:f66a8127-69a2-4b1b-9e9e-dd821c1a1b57 nodeName:}" failed. No retries permitted until 2026-09-14 13:32:05.397672187 +0000 UTC m=+2018.689006015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v" (UID: "f66a8127-69a2-4b1b-9e9e-dd821c1a1b57") : secret "maas-api-e2e-ipp-cleanup-8e2f31-serving-cert" not found
Sep 14 13:32:04.898571 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.898548    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:04.900828 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.900809    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:04.909282 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.909257    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xplh7\" (UniqueName: \"kubernetes.io/projected/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-kube-api-access-xplh7\") pod \"maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:04.911486 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:04.911466    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:05.265729 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:05.265699    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:05.337706 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:05.337674    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:05.401971 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:05.401931    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:05.402153 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:32:05.402104    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-serving-cert: secret "maas-api-e2e-ipp-cleanup-8e2f31-serving-cert" not found
Sep 14 13:32:05.402229 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:32:05.402216    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-maas-api-tls podName:f66a8127-69a2-4b1b-9e9e-dd821c1a1b57 nodeName:}" failed. No retries permitted until 2026-09-14 13:32:06.40219627 +0000 UTC m=+2019.693530067 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v" (UID: "f66a8127-69a2-4b1b-9e9e-dd821c1a1b57") : secret "maas-api-e2e-ipp-cleanup-8e2f31-serving-cert" not found
Sep 14 13:32:05.711108 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:05.711076    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:06.412515 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:06.412485    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:06.414829 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:06.414810    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:06.535916 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:06.535883    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:06.811431 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:06.811400    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"]
Sep 14 13:32:06.812842 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:32:06.812815    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf66a8127_69a2_4b1b_9e9e_dd821c1a1b57.slice/crio-ae1502a259806bc8b195d0499b1f756576ca07a9568eb4f946292f764e62f293 WatchSource:0}: Error finding container ae1502a259806bc8b195d0499b1f756576ca07a9568eb4f946292f764e62f293: Status 404 returned error can't find the container with id ae1502a259806bc8b195d0499b1f756576ca07a9568eb4f946292f764e62f293
Sep 14 13:32:07.676290 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:07.676252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v" event={"ID":"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57","Type":"ContainerStarted","Data":"61c8cb5764b386adda190d912d21006c5be77ee9a4be3d1e14f2bf4a68918bd5"}
Sep 14 13:32:07.676691 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:07.676330    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v" event={"ID":"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57","Type":"ContainerStarted","Data":"ae1502a259806bc8b195d0499b1f756576ca07a9568eb4f946292f764e62f293"}
Sep 14 13:32:07.676691 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:07.676420    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:07.706167 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:07.706108    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v" podStartSLOduration=3.240503972 podStartE2EDuration="3.706088831s" totalImagesPullingTime="465.584859ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:32:07.702970603 +0000 UTC m=+2020.994304420" watchObservedRunningTime="2026-09-14 13:32:07.706088831 +0000 UTC m=+2020.997422648"
Sep 14 13:32:11.155573 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:11.155517    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:32:13.686699 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:13.686674    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:26.536390 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:26.536329    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:26.636421 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:26.636389    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:27.428625 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:27.428590    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:28.056412 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:28.056379    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:28.725520 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:28.725487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:29.443698 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:29.443666    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:30.061573 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:30.061517    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:30.894820 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:30.894789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:31.531411 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:31.531376    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:33.004891 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.004861    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:33.554435 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.554397    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"]
Sep 14 13:32:33.554710 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.554681    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v" podUID="f66a8127-69a2-4b1b-9e9e-dd821c1a1b57" containerName="maas-api" containerID="cri-o://61c8cb5764b386adda190d912d21006c5be77ee9a4be3d1e14f2bf4a68918bd5" gracePeriod=30
Sep 14 13:32:33.589842 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.589797    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:33.675601 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.675573    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:33.682648 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.682617    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v" podUID="f66a8127-69a2-4b1b-9e9e-dd821c1a1b57" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.79:8443/health\": dial tcp 10.134.0.79:8443: connect: connection refused"
Sep 14 13:32:33.775217 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.775189    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:33.809822 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.809767    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:33.810596 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.810571    2581 generic.go:356] "Generic (PLEG): container finished" podID="f66a8127-69a2-4b1b-9e9e-dd821c1a1b57" containerID="61c8cb5764b386adda190d912d21006c5be77ee9a4be3d1e14f2bf4a68918bd5" exitCode=0
Sep 14 13:32:33.810714 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.810617    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v" event={"ID":"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57","Type":"ContainerDied","Data":"61c8cb5764b386adda190d912d21006c5be77ee9a4be3d1e14f2bf4a68918bd5"}
Sep 14 13:32:33.810714 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.810656    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v" event={"ID":"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57","Type":"ContainerDied","Data":"ae1502a259806bc8b195d0499b1f756576ca07a9568eb4f946292f764e62f293"}
Sep 14 13:32:33.810714 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.810676    2581 scope.go:117] "RemoveContainer" containerID="61c8cb5764b386adda190d912d21006c5be77ee9a4be3d1e14f2bf4a68918bd5"
Sep 14 13:32:33.819237 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.819211    2581 scope.go:117] "RemoveContainer" containerID="61c8cb5764b386adda190d912d21006c5be77ee9a4be3d1e14f2bf4a68918bd5"
Sep 14 13:32:33.819496 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:32:33.819478    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61c8cb5764b386adda190d912d21006c5be77ee9a4be3d1e14f2bf4a68918bd5\": container with ID starting with 61c8cb5764b386adda190d912d21006c5be77ee9a4be3d1e14f2bf4a68918bd5 not found: ID does not exist" containerID="61c8cb5764b386adda190d912d21006c5be77ee9a4be3d1e14f2bf4a68918bd5"
Sep 14 13:32:33.819552 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.819505    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61c8cb5764b386adda190d912d21006c5be77ee9a4be3d1e14f2bf4a68918bd5"} err="failed to get container status \"61c8cb5764b386adda190d912d21006c5be77ee9a4be3d1e14f2bf4a68918bd5\": rpc error: code = NotFound desc = could not find container \"61c8cb5764b386adda190d912d21006c5be77ee9a4be3d1e14f2bf4a68918bd5\": container with ID starting with 61c8cb5764b386adda190d912d21006c5be77ee9a4be3d1e14f2bf4a68918bd5 not found: ID does not exist"
Sep 14 13:32:33.856813 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.856786    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-maas-api-tls\") pod \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") "
Sep 14 13:32:33.856962 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.856879    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-metrics-certs\") pod \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") "
Sep 14 13:32:33.856962 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.856938    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xplh7\" (UniqueName: \"kubernetes.io/projected/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-kube-api-access-xplh7\") pod \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") "
Sep 14 13:32:33.857078 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.856968    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-openshift-service-ca\") pod \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\" (UID: \"f66a8127-69a2-4b1b-9e9e-dd821c1a1b57\") "
Sep 14 13:32:33.857335 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.857311    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f66a8127-69a2-4b1b-9e9e-dd821c1a1b57" (UID: "f66a8127-69a2-4b1b-9e9e-dd821c1a1b57"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:32:33.858825 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.858786    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f66a8127-69a2-4b1b-9e9e-dd821c1a1b57" (UID: "f66a8127-69a2-4b1b-9e9e-dd821c1a1b57"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:32:33.858915 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.858842    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f66a8127-69a2-4b1b-9e9e-dd821c1a1b57" (UID: "f66a8127-69a2-4b1b-9e9e-dd821c1a1b57"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:32:33.858915 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.858882    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-kube-api-access-xplh7" (OuterVolumeSpecName: "kube-api-access-xplh7") pod "f66a8127-69a2-4b1b-9e9e-dd821c1a1b57" (UID: "f66a8127-69a2-4b1b-9e9e-dd821c1a1b57"). InnerVolumeSpecName "kube-api-access-xplh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:32:33.876974 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.876951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:33.958151 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.958119    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-maas-api-tls\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:32:33.958151 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.958148    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-metrics-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:32:33.958341 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.958163    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xplh7\" (UniqueName: \"kubernetes.io/projected/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-kube-api-access-xplh7\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:32:33.958341 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:33.958177    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57-openshift-service-ca\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:32:34.144641 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:34.144611    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:34.248152 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:34.248122    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:34.817172 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:34.817138    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"
Sep 14 13:32:34.838382 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:34.838354    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"]
Sep 14 13:32:34.842880 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:34.842860    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-8e2f31-7845954dbb-jwv4v"]
Sep 14 13:32:35.116047 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:35.116017    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:35.246019 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:35.245984    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f66a8127-69a2-4b1b-9e9e-dd821c1a1b57" path="/var/lib/kubelet/pods/f66a8127-69a2-4b1b-9e9e-dd821c1a1b57/volumes"
Sep 14 13:32:35.785452 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:35.785420    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:36.428143 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:36.428114    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:37.103754 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:37.103723    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:38.564812 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:38.564774    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:32:40.036196 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:40.036152    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:40.148128 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:40.148074    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:40.307068 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:40.306979    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:44.035684 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:44.035648    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:44.794785 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:44.794752    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:45.514304 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:45.514270    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:46.217899 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:46.217873    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:46.871020 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:46.870985    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:47.603870 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:47.603838    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:56.759300 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:56.759222    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:32:59.194884 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.194856    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:59.222958 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.222927    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"]
Sep 14 13:32:59.223209 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.223185    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8" podUID="5860a73f-c28e-4c62-8611-cc126fae5cae" containerName="payload-processing" containerID="cri-o://68bbac89327e4f16a51e075ba4f410c7855d17088f77f3a0a9d82eb82088c120" gracePeriod=30
Sep 14 13:32:59.261575 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.261520    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:59.322110 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.322072    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:59.387357 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.387327    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:59.475302 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.475279    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"
Sep 14 13:32:59.587075 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.587051    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5860a73f-c28e-4c62-8611-cc126fae5cae-plugins-config-volume\") pod \"5860a73f-c28e-4c62-8611-cc126fae5cae\" (UID: \"5860a73f-c28e-4c62-8611-cc126fae5cae\") "
Sep 14 13:32:59.587226 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.587086    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tl7hs\" (UniqueName: \"kubernetes.io/projected/5860a73f-c28e-4c62-8611-cc126fae5cae-kube-api-access-tl7hs\") pod \"5860a73f-c28e-4c62-8611-cc126fae5cae\" (UID: \"5860a73f-c28e-4c62-8611-cc126fae5cae\") "
Sep 14 13:32:59.587400 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.587379    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5860a73f-c28e-4c62-8611-cc126fae5cae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5860a73f-c28e-4c62-8611-cc126fae5cae" (UID: "5860a73f-c28e-4c62-8611-cc126fae5cae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:32:59.589102 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.589081    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5860a73f-c28e-4c62-8611-cc126fae5cae-kube-api-access-tl7hs" (OuterVolumeSpecName: "kube-api-access-tl7hs") pod "5860a73f-c28e-4c62-8611-cc126fae5cae" (UID: "5860a73f-c28e-4c62-8611-cc126fae5cae"). InnerVolumeSpecName "kube-api-access-tl7hs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:32:59.688526 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.688492    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5860a73f-c28e-4c62-8611-cc126fae5cae-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:32:59.688526 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.688521    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl7hs\" (UniqueName: \"kubernetes.io/projected/5860a73f-c28e-4c62-8611-cc126fae5cae-kube-api-access-tl7hs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:32:59.743947 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.743856    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:59.844089 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.844058    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:59.938225 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.938198    2581 generic.go:356] "Generic (PLEG): container finished" podID="5860a73f-c28e-4c62-8611-cc126fae5cae" containerID="68bbac89327e4f16a51e075ba4f410c7855d17088f77f3a0a9d82eb82088c120" exitCode=0
Sep 14 13:32:59.938389 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.938230    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8" event={"ID":"5860a73f-c28e-4c62-8611-cc126fae5cae","Type":"ContainerDied","Data":"68bbac89327e4f16a51e075ba4f410c7855d17088f77f3a0a9d82eb82088c120"}
Sep 14 13:32:59.938389 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.938244    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"
Sep 14 13:32:59.938389 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.938271    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8" event={"ID":"5860a73f-c28e-4c62-8611-cc126fae5cae","Type":"ContainerDied","Data":"6ca0f3a852dbb3ee631f9678d7db11a0365164a9de6a1d22b4d26d23d9488858"}
Sep 14 13:32:59.938389 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.938293    2581 scope.go:117] "RemoveContainer" containerID="68bbac89327e4f16a51e075ba4f410c7855d17088f77f3a0a9d82eb82088c120"
Sep 14 13:32:59.946830 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.946810    2581 scope.go:117] "RemoveContainer" containerID="68bbac89327e4f16a51e075ba4f410c7855d17088f77f3a0a9d82eb82088c120"
Sep 14 13:32:59.947093 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:32:59.947076    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68bbac89327e4f16a51e075ba4f410c7855d17088f77f3a0a9d82eb82088c120\": container with ID starting with 68bbac89327e4f16a51e075ba4f410c7855d17088f77f3a0a9d82eb82088c120 not found: ID does not exist" containerID="68bbac89327e4f16a51e075ba4f410c7855d17088f77f3a0a9d82eb82088c120"
Sep 14 13:32:59.947154 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.947102    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68bbac89327e4f16a51e075ba4f410c7855d17088f77f3a0a9d82eb82088c120"} err="failed to get container status \"68bbac89327e4f16a51e075ba4f410c7855d17088f77f3a0a9d82eb82088c120\": rpc error: code = NotFound desc = could not find container \"68bbac89327e4f16a51e075ba4f410c7855d17088f77f3a0a9d82eb82088c120\": container with ID starting with 68bbac89327e4f16a51e075ba4f410c7855d17088f77f3a0a9d82eb82088c120 not found: ID does not exist"
Sep 14 13:32:59.959903 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.959881    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"]
Sep 14 13:32:59.965632 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:32:59.965612    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c87a95-75787d7c55-ttbc8"]
Sep 14 13:33:00.479936 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:00.479907    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:01.143859 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:01.143831    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:01.245362 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:01.245324    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5860a73f-c28e-4c62-8611-cc126fae5cae" path="/var/lib/kubelet/pods/5860a73f-c28e-4c62-8611-cc126fae5cae/volumes"
Sep 14 13:33:01.839882 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:01.839842    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:04.818416 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:04.818386    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:09.052602 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:09.052566    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:33:22.436561 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:22.436513    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:27.317857 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:27.317832    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:33:27.323371 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:27.323349    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:33:33.230406 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:33.230370    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:33.327733 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:33.327698    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:35.604567 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:35.604510    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:40.957771 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:40.957736    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:41.904417 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:41.904383    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:47.922838 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:47.922789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:47.990907 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:47.990875    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:48.205303 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:48.205225    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:48.838019 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:48.837985    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:49.645299 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:49.645264    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:50.443371 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:50.443343    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:51.067842 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:51.067810    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:51.807547 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:51.807486    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:59.037921 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.037888    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:59.059174 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.059142    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"]
Sep 14 13:33:59.059439 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.059414    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr" podUID="21c13b16-51cf-4a74-81f0-2e9e4296dfa0" containerName="payload-processing" containerID="cri-o://a2c8b93fc3c607527932de1629d1484687dde7b954495b051d3b5db15d5977fb" gracePeriod=30
Sep 14 13:33:59.137826 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.137794    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:59.215088 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.215052    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:59.224704 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.224660    2581 generic.go:356] "Generic (PLEG): container finished" podID="21c13b16-51cf-4a74-81f0-2e9e4296dfa0" containerID="a2c8b93fc3c607527932de1629d1484687dde7b954495b051d3b5db15d5977fb" exitCode=0
Sep 14 13:33:59.224849 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.224730    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr" event={"ID":"21c13b16-51cf-4a74-81f0-2e9e4296dfa0","Type":"ContainerDied","Data":"a2c8b93fc3c607527932de1629d1484687dde7b954495b051d3b5db15d5977fb"}
Sep 14 13:33:59.277659 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.277627    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:59.306849 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.306792    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"
Sep 14 13:33:59.420204 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.420174    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21c13b16-51cf-4a74-81f0-2e9e4296dfa0-plugins-config-volume\") pod \"21c13b16-51cf-4a74-81f0-2e9e4296dfa0\" (UID: \"21c13b16-51cf-4a74-81f0-2e9e4296dfa0\") "
Sep 14 13:33:59.420365 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.420213    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brv6n\" (UniqueName: \"kubernetes.io/projected/21c13b16-51cf-4a74-81f0-2e9e4296dfa0-kube-api-access-brv6n\") pod \"21c13b16-51cf-4a74-81f0-2e9e4296dfa0\" (UID: \"21c13b16-51cf-4a74-81f0-2e9e4296dfa0\") "
Sep 14 13:33:59.420597 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.420574    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21c13b16-51cf-4a74-81f0-2e9e4296dfa0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "21c13b16-51cf-4a74-81f0-2e9e4296dfa0" (UID: "21c13b16-51cf-4a74-81f0-2e9e4296dfa0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:33:59.422470 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.422446    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21c13b16-51cf-4a74-81f0-2e9e4296dfa0-kube-api-access-brv6n" (OuterVolumeSpecName: "kube-api-access-brv6n") pod "21c13b16-51cf-4a74-81f0-2e9e4296dfa0" (UID: "21c13b16-51cf-4a74-81f0-2e9e4296dfa0"). InnerVolumeSpecName "kube-api-access-brv6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:33:59.520829 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.520796    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21c13b16-51cf-4a74-81f0-2e9e4296dfa0-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:33:59.520829 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.520825    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brv6n\" (UniqueName: \"kubernetes.io/projected/21c13b16-51cf-4a74-81f0-2e9e4296dfa0-kube-api-access-brv6n\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:33:59.582571 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.582487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:59.682343 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:33:59.682300    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:00.230216 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:00.230187    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:00.231371 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:00.231353    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"
Sep 14 13:34:00.231498 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:00.231365    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr" event={"ID":"21c13b16-51cf-4a74-81f0-2e9e4296dfa0","Type":"ContainerDied","Data":"94cc24326c86dd822d74a75921e40689f05d0d92db34854c3d75d4bbbf9dfa4a"}
Sep 14 13:34:00.231498 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:00.231419    2581 scope.go:117] "RemoveContainer" containerID="a2c8b93fc3c607527932de1629d1484687dde7b954495b051d3b5db15d5977fb"
Sep 14 13:34:00.254661 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:00.254636    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"]
Sep 14 13:34:00.259576 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:00.259554    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-27ae76-6f7f7cb887-tsvkr"]
Sep 14 13:34:01.026313 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:01.026281    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:01.244939 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:01.244909    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21c13b16-51cf-4a74-81f0-2e9e4296dfa0" path="/var/lib/kubelet/pods/21c13b16-51cf-4a74-81f0-2e9e4296dfa0/volumes"
Sep 14 13:34:02.258924 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:02.258877    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:34:04.809070 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:04.809033    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:04.910573 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:04.910522    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:05.752573 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:05.752526    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:05.939912 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:05.939885    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:06.007806 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:06.007732    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:06.621007 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:06.620977    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:07.225492 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:07.225457    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:08.021922 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:08.021891    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:08.733917 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:08.733882    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:16.806627 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:16.806596    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"]
Sep 14 13:34:16.809135 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:16.806862    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7" podUID="3e34f45e-5dca-48cb-8f45-9a9d84be39a6" containerName="maas-api" containerID="cri-o://277df7da569fb95615844d4fd9a296c1cea86ad3e5ffbf028f36d332f1367d94" gracePeriod=30
Sep 14 13:34:16.864880 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:16.864850    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"]
Sep 14 13:34:16.865095 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:16.865071    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn" podUID="d6073121-67f4-4240-827f-1060d208c62c" containerName="payload-pre-processing" containerID="cri-o://a88811301e746bb133932d9eff66df097a987cc99dc0344928f8280d32b4409c" gracePeriod=30
Sep 14 13:34:16.907969 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:16.907940    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:17.006084 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.006041    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:17.102883 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.102859    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:34:17.111062 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.111039    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:17.133577 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.133552    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"
Sep 14 13:34:17.169429 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.169402    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-openshift-service-ca\") pod \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") "
Sep 14 13:34:17.169589 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.169451    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z78h7\" (UniqueName: \"kubernetes.io/projected/d6073121-67f4-4240-827f-1060d208c62c-kube-api-access-z78h7\") pod \"d6073121-67f4-4240-827f-1060d208c62c\" (UID: \"d6073121-67f4-4240-827f-1060d208c62c\") "
Sep 14 13:34:17.169589 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.169470    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6073121-67f4-4240-827f-1060d208c62c-plugins-config-volume\") pod \"d6073121-67f4-4240-827f-1060d208c62c\" (UID: \"d6073121-67f4-4240-827f-1060d208c62c\") "
Sep 14 13:34:17.169589 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.169561    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-872xk\" (UniqueName: \"kubernetes.io/projected/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-kube-api-access-872xk\") pod \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") "
Sep 14 13:34:17.169781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.169597    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-maas-api-tls\") pod \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") "
Sep 14 13:34:17.169781 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.169627    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-metrics-certs\") pod \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\" (UID: \"3e34f45e-5dca-48cb-8f45-9a9d84be39a6\") "
Sep 14 13:34:17.169947 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.169906    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6073121-67f4-4240-827f-1060d208c62c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d6073121-67f4-4240-827f-1060d208c62c" (UID: "d6073121-67f4-4240-827f-1060d208c62c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:34:17.170096 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.170058    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3e34f45e-5dca-48cb-8f45-9a9d84be39a6" (UID: "3e34f45e-5dca-48cb-8f45-9a9d84be39a6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:34:17.171979 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.171897    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3e34f45e-5dca-48cb-8f45-9a9d84be39a6" (UID: "3e34f45e-5dca-48cb-8f45-9a9d84be39a6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:34:17.172214 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.172191    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-kube-api-access-872xk" (OuterVolumeSpecName: "kube-api-access-872xk") pod "3e34f45e-5dca-48cb-8f45-9a9d84be39a6" (UID: "3e34f45e-5dca-48cb-8f45-9a9d84be39a6"). InnerVolumeSpecName "kube-api-access-872xk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:34:17.172307 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.172224    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6073121-67f4-4240-827f-1060d208c62c-kube-api-access-z78h7" (OuterVolumeSpecName: "kube-api-access-z78h7") pod "d6073121-67f4-4240-827f-1060d208c62c" (UID: "d6073121-67f4-4240-827f-1060d208c62c"). InnerVolumeSpecName "kube-api-access-z78h7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:34:17.172367 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.172333    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3e34f45e-5dca-48cb-8f45-9a9d84be39a6" (UID: "3e34f45e-5dca-48cb-8f45-9a9d84be39a6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:34:17.271096 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.271068    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-openshift-service-ca\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:34:17.271096 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.271091    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z78h7\" (UniqueName: \"kubernetes.io/projected/d6073121-67f4-4240-827f-1060d208c62c-kube-api-access-z78h7\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:34:17.271252 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.271103    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6073121-67f4-4240-827f-1060d208c62c-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:34:17.271252 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.271111    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-872xk\" (UniqueName: \"kubernetes.io/projected/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-kube-api-access-872xk\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:34:17.271252 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.271120    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-maas-api-tls\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:34:17.271252 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.271138    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e34f45e-5dca-48cb-8f45-9a9d84be39a6-metrics-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:34:17.319566 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.319445    2581 generic.go:356] "Generic (PLEG): container finished" podID="3e34f45e-5dca-48cb-8f45-9a9d84be39a6" containerID="277df7da569fb95615844d4fd9a296c1cea86ad3e5ffbf028f36d332f1367d94" exitCode=0
Sep 14 13:34:17.319566 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.319516    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"
Sep 14 13:34:17.319566 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.319545    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7" event={"ID":"3e34f45e-5dca-48cb-8f45-9a9d84be39a6","Type":"ContainerDied","Data":"277df7da569fb95615844d4fd9a296c1cea86ad3e5ffbf028f36d332f1367d94"}
Sep 14 13:34:17.319859 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.319643    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7" event={"ID":"3e34f45e-5dca-48cb-8f45-9a9d84be39a6","Type":"ContainerDied","Data":"cbc3b00b4ffcb2b3e6f59b0a35eaa206c11be6b11a3cd0e66c67af3969bca7e0"}
Sep 14 13:34:17.319859 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.319671    2581 scope.go:117] "RemoveContainer" containerID="277df7da569fb95615844d4fd9a296c1cea86ad3e5ffbf028f36d332f1367d94"
Sep 14 13:34:17.321062 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.321036    2581 generic.go:356] "Generic (PLEG): container finished" podID="d6073121-67f4-4240-827f-1060d208c62c" containerID="a88811301e746bb133932d9eff66df097a987cc99dc0344928f8280d32b4409c" exitCode=0
Sep 14 13:34:17.321230 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.321086    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"
Sep 14 13:34:17.321230 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.321101    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn" event={"ID":"d6073121-67f4-4240-827f-1060d208c62c","Type":"ContainerDied","Data":"a88811301e746bb133932d9eff66df097a987cc99dc0344928f8280d32b4409c"}
Sep 14 13:34:17.321230 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.321137    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn" event={"ID":"d6073121-67f4-4240-827f-1060d208c62c","Type":"ContainerDied","Data":"a16b6476c9571cd048bd494b1522ae879aef32506e2eee753924dce99b7075c9"}
Sep 14 13:34:17.328577 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.328561    2581 scope.go:117] "RemoveContainer" containerID="277df7da569fb95615844d4fd9a296c1cea86ad3e5ffbf028f36d332f1367d94"
Sep 14 13:34:17.328840 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:34:17.328822    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"277df7da569fb95615844d4fd9a296c1cea86ad3e5ffbf028f36d332f1367d94\": container with ID starting with 277df7da569fb95615844d4fd9a296c1cea86ad3e5ffbf028f36d332f1367d94 not found: ID does not exist" containerID="277df7da569fb95615844d4fd9a296c1cea86ad3e5ffbf028f36d332f1367d94"
Sep 14 13:34:17.328905 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.328846    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"277df7da569fb95615844d4fd9a296c1cea86ad3e5ffbf028f36d332f1367d94"} err="failed to get container status \"277df7da569fb95615844d4fd9a296c1cea86ad3e5ffbf028f36d332f1367d94\": rpc error: code = NotFound desc = could not find container \"277df7da569fb95615844d4fd9a296c1cea86ad3e5ffbf028f36d332f1367d94\": container with ID starting with 277df7da569fb95615844d4fd9a296c1cea86ad3e5ffbf028f36d332f1367d94 not found: ID does not exist"
Sep 14 13:34:17.328905 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.328861    2581 scope.go:117] "RemoveContainer" containerID="a88811301e746bb133932d9eff66df097a987cc99dc0344928f8280d32b4409c"
Sep 14 13:34:17.336624 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.336604    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"]
Sep 14 13:34:17.336736 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.336625    2581 scope.go:117] "RemoveContainer" containerID="a88811301e746bb133932d9eff66df097a987cc99dc0344928f8280d32b4409c"
Sep 14 13:34:17.336935 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:34:17.336916    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a88811301e746bb133932d9eff66df097a987cc99dc0344928f8280d32b4409c\": container with ID starting with a88811301e746bb133932d9eff66df097a987cc99dc0344928f8280d32b4409c not found: ID does not exist" containerID="a88811301e746bb133932d9eff66df097a987cc99dc0344928f8280d32b4409c"
Sep 14 13:34:17.336980 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.336947    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a88811301e746bb133932d9eff66df097a987cc99dc0344928f8280d32b4409c"} err="failed to get container status \"a88811301e746bb133932d9eff66df097a987cc99dc0344928f8280d32b4409c\": rpc error: code = NotFound desc = could not find container \"a88811301e746bb133932d9eff66df097a987cc99dc0344928f8280d32b4409c\": container with ID starting with a88811301e746bb133932d9eff66df097a987cc99dc0344928f8280d32b4409c not found: ID does not exist"
Sep 14 13:34:17.341966 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.341945    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-14f3af-65db7cfc58-g2tt7"]
Sep 14 13:34:17.354545 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.354511    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"]
Sep 14 13:34:17.358713 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.358692    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14f3af-58c4d659db-nldwn"]
Sep 14 13:34:17.413626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.413597    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:17.514053 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:17.514015    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:18.111303 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:18.111251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:18.758825 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:18.758790    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:34:19.245964 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:19.245932    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e34f45e-5dca-48cb-8f45-9a9d84be39a6" path="/var/lib/kubelet/pods/3e34f45e-5dca-48cb-8f45-9a9d84be39a6/volumes"
Sep 14 13:34:19.246334 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:19.246303    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6073121-67f4-4240-827f-1060d208c62c" path="/var/lib/kubelet/pods/d6073121-67f4-4240-827f-1060d208c62c/volumes"
Sep 14 13:34:23.506573 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:23.506525    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:23.606648 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:23.606620    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:23.762392 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:23.762319    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:24.895295 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:24.895260    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:25.867752 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:25.867724    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:26.216303 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:26.216225    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:34.238659 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:34.238630    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:34.807448 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:34.807412    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-eff5e8-29823210-44x58"]
Sep 14 13:34:34.810802 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:34.810777    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-eff5e8-29823210-44x58"]
Sep 14 13:34:34.827160 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:34.827134    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"]
Sep 14 13:34:34.827337 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:34.827314    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv" podUID="31459634-487d-4350-b79e-67dda73fc43f" containerName="payload-processing" containerID="cri-o://5c5027023f11f5b5c452d4addf8afa8591cdde65f3311c371babbc9dccfa6322" gracePeriod=30
Sep 14 13:34:35.075519 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.075488    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"
Sep 14 13:34:35.228574 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.228513    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31459634-487d-4350-b79e-67dda73fc43f-plugins-config-volume\") pod \"31459634-487d-4350-b79e-67dda73fc43f\" (UID: \"31459634-487d-4350-b79e-67dda73fc43f\") "
Sep 14 13:34:35.228574 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.228583    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vk9h2\" (UniqueName: \"kubernetes.io/projected/31459634-487d-4350-b79e-67dda73fc43f-kube-api-access-vk9h2\") pod \"31459634-487d-4350-b79e-67dda73fc43f\" (UID: \"31459634-487d-4350-b79e-67dda73fc43f\") "
Sep 14 13:34:35.228876 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.228851    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31459634-487d-4350-b79e-67dda73fc43f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "31459634-487d-4350-b79e-67dda73fc43f" (UID: "31459634-487d-4350-b79e-67dda73fc43f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:34:35.230599 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.230579    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31459634-487d-4350-b79e-67dda73fc43f-kube-api-access-vk9h2" (OuterVolumeSpecName: "kube-api-access-vk9h2") pod "31459634-487d-4350-b79e-67dda73fc43f" (UID: "31459634-487d-4350-b79e-67dda73fc43f"). InnerVolumeSpecName "kube-api-access-vk9h2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:34:35.245270 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.245243    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b70d3dba-151d-4e33-8da7-08b51bf64559" path="/var/lib/kubelet/pods/b70d3dba-151d-4e33-8da7-08b51bf64559/volumes"
Sep 14 13:34:35.330211 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.330141    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31459634-487d-4350-b79e-67dda73fc43f-plugins-config-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:34:35.330211 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.330168    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vk9h2\" (UniqueName: \"kubernetes.io/projected/31459634-487d-4350-b79e-67dda73fc43f-kube-api-access-vk9h2\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:34:35.406723 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.406686    2581 generic.go:356] "Generic (PLEG): container finished" podID="31459634-487d-4350-b79e-67dda73fc43f" containerID="5c5027023f11f5b5c452d4addf8afa8591cdde65f3311c371babbc9dccfa6322" exitCode=0
Sep 14 13:34:35.406863 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.406740    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"
Sep 14 13:34:35.406863 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.406769    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv" event={"ID":"31459634-487d-4350-b79e-67dda73fc43f","Type":"ContainerDied","Data":"5c5027023f11f5b5c452d4addf8afa8591cdde65f3311c371babbc9dccfa6322"}
Sep 14 13:34:35.406863 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.406818    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv" event={"ID":"31459634-487d-4350-b79e-67dda73fc43f","Type":"ContainerDied","Data":"e2dba9dcdde30f7fa475f2d660f08df991471dd6a50602dee9a1199f6474ca52"}
Sep 14 13:34:35.406863 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.406841    2581 scope.go:117] "RemoveContainer" containerID="5c5027023f11f5b5c452d4addf8afa8591cdde65f3311c371babbc9dccfa6322"
Sep 14 13:34:35.415468 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.415454    2581 scope.go:117] "RemoveContainer" containerID="5c5027023f11f5b5c452d4addf8afa8591cdde65f3311c371babbc9dccfa6322"
Sep 14 13:34:35.415709 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:34:35.415692    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c5027023f11f5b5c452d4addf8afa8591cdde65f3311c371babbc9dccfa6322\": container with ID starting with 5c5027023f11f5b5c452d4addf8afa8591cdde65f3311c371babbc9dccfa6322 not found: ID does not exist" containerID="5c5027023f11f5b5c452d4addf8afa8591cdde65f3311c371babbc9dccfa6322"
Sep 14 13:34:35.415787 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.415715    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c5027023f11f5b5c452d4addf8afa8591cdde65f3311c371babbc9dccfa6322"} err="failed to get container status \"5c5027023f11f5b5c452d4addf8afa8591cdde65f3311c371babbc9dccfa6322\": rpc error: code = NotFound desc = could not find container \"5c5027023f11f5b5c452d4addf8afa8591cdde65f3311c371babbc9dccfa6322\": container with ID starting with 5c5027023f11f5b5c452d4addf8afa8591cdde65f3311c371babbc9dccfa6322 not found: ID does not exist"
Sep 14 13:34:35.423976 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.423939    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"]
Sep 14 13:34:35.425930 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.425912    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-eff5e8-558c9dbc79-2qbkv"]
Sep 14 13:34:35.818764 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:35.818731    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:37.166046 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:37.166012    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:34:37.246353 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:37.246325    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31459634-487d-4350-b79e-67dda73fc43f" path="/var/lib/kubelet/pods/31459634-487d-4350-b79e-67dda73fc43f/volumes"
Sep 14 13:34:41.320036 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:41.320004    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:41.420866 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:41.420836    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:41.524946 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:41.524919    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:45.858995 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:34:45.858960    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:35:42.419553 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:35:42.419490    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:35:42.517752 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:35:42.517718    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:35:46.061233 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:35:46.061201    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:36:05.956474 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:36:05.956397    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:36:18.306620 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:36:18.306589    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:36:19.924169 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:36:19.924135    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:36:27.618486 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:36:27.618455    2581 scope.go:117] "RemoveContainer" containerID="13bb883b4993aea0b92df592876629b3c533b74b59a4879241d131504d77a0b0"
Sep 14 13:36:48.148902 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:36:48.148870    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:36:55.753289 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:36:55.753251    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:37:02.456846 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:02.456814    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:37:08.755423 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:08.755387    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:37:11.707814 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:11.707776    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:37:11.775404 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:11.775370    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:37:23.252273 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:23.252238    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:37:32.458471 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:32.458438    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:37:34.312262 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:34.312227    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"]
Sep 14 13:37:34.312734 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:34.312431    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm" podUID="45772d7c-5790-4642-940a-335038b9560d" containerName="manager" containerID="cri-o://95998d2dac2e70525ea06b732475d3428a684578e6144d2cc7cf26809fb28ea3" gracePeriod=10
Sep 14 13:37:35.953228 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:35.953204    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"
Sep 14 13:37:36.094792 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.094707    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/45772d7c-5790-4642-940a-335038b9560d-extensions-socket-volume\") pod \"45772d7c-5790-4642-940a-335038b9560d\" (UID: \"45772d7c-5790-4642-940a-335038b9560d\") "
Sep 14 13:37:36.094792 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.094754    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pj2kr\" (UniqueName: \"kubernetes.io/projected/45772d7c-5790-4642-940a-335038b9560d-kube-api-access-pj2kr\") pod \"45772d7c-5790-4642-940a-335038b9560d\" (UID: \"45772d7c-5790-4642-940a-335038b9560d\") "
Sep 14 13:37:36.095115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.095089    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45772d7c-5790-4642-940a-335038b9560d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "45772d7c-5790-4642-940a-335038b9560d" (UID: "45772d7c-5790-4642-940a-335038b9560d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:37:36.096816 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.096794    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45772d7c-5790-4642-940a-335038b9560d-kube-api-access-pj2kr" (OuterVolumeSpecName: "kube-api-access-pj2kr") pod "45772d7c-5790-4642-940a-335038b9560d" (UID: "45772d7c-5790-4642-940a-335038b9560d"). InnerVolumeSpecName "kube-api-access-pj2kr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:37:36.196009 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.195961    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj2kr\" (UniqueName: \"kubernetes.io/projected/45772d7c-5790-4642-940a-335038b9560d-kube-api-access-pj2kr\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:37:36.196009 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.196005    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/45772d7c-5790-4642-940a-335038b9560d-extensions-socket-volume\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:37:36.225876 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.225846    2581 generic.go:356] "Generic (PLEG): container finished" podID="45772d7c-5790-4642-940a-335038b9560d" containerID="95998d2dac2e70525ea06b732475d3428a684578e6144d2cc7cf26809fb28ea3" exitCode=0
Sep 14 13:37:36.226008 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.225888    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"
Sep 14 13:37:36.226008 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.225935    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm" event={"ID":"45772d7c-5790-4642-940a-335038b9560d","Type":"ContainerDied","Data":"95998d2dac2e70525ea06b732475d3428a684578e6144d2cc7cf26809fb28ea3"}
Sep 14 13:37:36.226089 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.226012    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm" event={"ID":"45772d7c-5790-4642-940a-335038b9560d","Type":"ContainerDied","Data":"baf3b781798ac244003bd4f97c9546f0acc075f9cb04817a46423337cfbb1f22"}
Sep 14 13:37:36.226089 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.226037    2581 scope.go:117] "RemoveContainer" containerID="95998d2dac2e70525ea06b732475d3428a684578e6144d2cc7cf26809fb28ea3"
Sep 14 13:37:36.234878 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.234863    2581 scope.go:117] "RemoveContainer" containerID="95998d2dac2e70525ea06b732475d3428a684578e6144d2cc7cf26809fb28ea3"
Sep 14 13:37:36.235174 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:37:36.235157    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95998d2dac2e70525ea06b732475d3428a684578e6144d2cc7cf26809fb28ea3\": container with ID starting with 95998d2dac2e70525ea06b732475d3428a684578e6144d2cc7cf26809fb28ea3 not found: ID does not exist" containerID="95998d2dac2e70525ea06b732475d3428a684578e6144d2cc7cf26809fb28ea3"
Sep 14 13:37:36.235223 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.235185    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95998d2dac2e70525ea06b732475d3428a684578e6144d2cc7cf26809fb28ea3"} err="failed to get container status \"95998d2dac2e70525ea06b732475d3428a684578e6144d2cc7cf26809fb28ea3\": rpc error: code = NotFound desc = could not find container \"95998d2dac2e70525ea06b732475d3428a684578e6144d2cc7cf26809fb28ea3\": container with ID starting with 95998d2dac2e70525ea06b732475d3428a684578e6144d2cc7cf26809fb28ea3 not found: ID does not exist"
Sep 14 13:37:36.246716 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.246693    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"]
Sep 14 13:37:36.251571 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:36.251552    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w6hzm"]
Sep 14 13:37:37.245021 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:37:37.244992    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45772d7c-5790-4642-940a-335038b9560d" path="/var/lib/kubelet/pods/45772d7c-5790-4642-940a-335038b9560d/volumes"
Sep 14 13:38:27.349542 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:27.349512    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:38:27.356366 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:27.356348    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d8mhr_9e5cd353-35cb-4693-a0e3-a49a546d8528/ovn-acl-logging/0.log"
Sep 14 13:38:34.996764 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.996729    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq"]
Sep 14 13:38:34.997133 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997051    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31459634-487d-4350-b79e-67dda73fc43f" containerName="payload-processing"
Sep 14 13:38:34.997133 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997062    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="31459634-487d-4350-b79e-67dda73fc43f" containerName="payload-processing"
Sep 14 13:38:34.997133 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997076    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e34f45e-5dca-48cb-8f45-9a9d84be39a6" containerName="maas-api"
Sep 14 13:38:34.997133 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997081    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e34f45e-5dca-48cb-8f45-9a9d84be39a6" containerName="maas-api"
Sep 14 13:38:34.997133 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997094    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45772d7c-5790-4642-940a-335038b9560d" containerName="manager"
Sep 14 13:38:34.997133 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997102    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="45772d7c-5790-4642-940a-335038b9560d" containerName="manager"
Sep 14 13:38:34.997133 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997111    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21c13b16-51cf-4a74-81f0-2e9e4296dfa0" containerName="payload-processing"
Sep 14 13:38:34.997133 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997116    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="21c13b16-51cf-4a74-81f0-2e9e4296dfa0" containerName="payload-processing"
Sep 14 13:38:34.997133 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997121    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6073121-67f4-4240-827f-1060d208c62c" containerName="payload-pre-processing"
Sep 14 13:38:34.997133 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997126    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6073121-67f4-4240-827f-1060d208c62c" containerName="payload-pre-processing"
Sep 14 13:38:34.997133 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997138    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5860a73f-c28e-4c62-8611-cc126fae5cae" containerName="payload-processing"
Sep 14 13:38:34.997465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997144    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5860a73f-c28e-4c62-8611-cc126fae5cae" containerName="payload-processing"
Sep 14 13:38:34.997465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997154    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f66a8127-69a2-4b1b-9e9e-dd821c1a1b57" containerName="maas-api"
Sep 14 13:38:34.997465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997159    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f66a8127-69a2-4b1b-9e9e-dd821c1a1b57" containerName="maas-api"
Sep 14 13:38:34.997465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997204    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f66a8127-69a2-4b1b-9e9e-dd821c1a1b57" containerName="maas-api"
Sep 14 13:38:34.997465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997211    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e34f45e-5dca-48cb-8f45-9a9d84be39a6" containerName="maas-api"
Sep 14 13:38:34.997465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997217    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6073121-67f4-4240-827f-1060d208c62c" containerName="payload-pre-processing"
Sep 14 13:38:34.997465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997223    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="45772d7c-5790-4642-940a-335038b9560d" containerName="manager"
Sep 14 13:38:34.997465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997228    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5860a73f-c28e-4c62-8611-cc126fae5cae" containerName="payload-processing"
Sep 14 13:38:34.997465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997233    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="21c13b16-51cf-4a74-81f0-2e9e4296dfa0" containerName="payload-processing"
Sep 14 13:38:34.997465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:34.997238    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="31459634-487d-4350-b79e-67dda73fc43f" containerName="payload-processing"
Sep 14 13:38:35.000033 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:35.000016    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq"
Sep 14 13:38:35.002495 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:35.002473    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fbs72\""
Sep 14 13:38:35.100847 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:35.100809    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq"]
Sep 14 13:38:35.114061 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:35.114028    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1b864c0-dcc0-4baf-a529-e0f94cdef80d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ntsnq\" (UID: \"f1b864c0-dcc0-4baf-a529-e0f94cdef80d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq"
Sep 14 13:38:35.114195 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:35.114154    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59ngh\" (UniqueName: \"kubernetes.io/projected/f1b864c0-dcc0-4baf-a529-e0f94cdef80d-kube-api-access-59ngh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ntsnq\" (UID: \"f1b864c0-dcc0-4baf-a529-e0f94cdef80d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq"
Sep 14 13:38:35.215332 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:35.215301    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1b864c0-dcc0-4baf-a529-e0f94cdef80d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ntsnq\" (UID: \"f1b864c0-dcc0-4baf-a529-e0f94cdef80d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq"
Sep 14 13:38:35.215471 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:35.215391    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59ngh\" (UniqueName: \"kubernetes.io/projected/f1b864c0-dcc0-4baf-a529-e0f94cdef80d-kube-api-access-59ngh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ntsnq\" (UID: \"f1b864c0-dcc0-4baf-a529-e0f94cdef80d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq"
Sep 14 13:38:35.215713 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:35.215690    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1b864c0-dcc0-4baf-a529-e0f94cdef80d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ntsnq\" (UID: \"f1b864c0-dcc0-4baf-a529-e0f94cdef80d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq"
Sep 14 13:38:35.224311 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:35.224289    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59ngh\" (UniqueName: \"kubernetes.io/projected/f1b864c0-dcc0-4baf-a529-e0f94cdef80d-kube-api-access-59ngh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ntsnq\" (UID: \"f1b864c0-dcc0-4baf-a529-e0f94cdef80d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq"
Sep 14 13:38:35.263660 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:35.263607    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:38:35.309738 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:35.309717    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq"
Sep 14 13:38:35.361120 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:35.361088    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:38:35.583666 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:35.583546    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq"]
Sep 14 13:38:35.586693 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:38:35.586667    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b864c0_dcc0_4baf_a529_e0f94cdef80d.slice/crio-29f6d68efa266c94875d88527474f25a64aa46a81a720a82879d722bb52391af WatchSource:0}: Error finding container 29f6d68efa266c94875d88527474f25a64aa46a81a720a82879d722bb52391af: Status 404 returned error can't find the container with id 29f6d68efa266c94875d88527474f25a64aa46a81a720a82879d722bb52391af
Sep 14 13:38:36.495082 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:36.495046    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq" event={"ID":"f1b864c0-dcc0-4baf-a529-e0f94cdef80d","Type":"ContainerStarted","Data":"cb82ab4b1d0373dd091a8fde9feb1f9592d04b533d503b6e1bea85272dcc18b6"}
Sep 14 13:38:36.495588 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:36.495095    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq" event={"ID":"f1b864c0-dcc0-4baf-a529-e0f94cdef80d","Type":"ContainerStarted","Data":"29f6d68efa266c94875d88527474f25a64aa46a81a720a82879d722bb52391af"}
Sep 14 13:38:36.495588 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:36.495161    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq"
Sep 14 13:38:36.518065 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:36.518012    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq" podStartSLOduration=2.517991937 podStartE2EDuration="2.517991937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:38:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:38:36.514722661 +0000 UTC m=+2409.806056478" watchObservedRunningTime="2026-09-14 13:38:36.517991937 +0000 UTC m=+2409.809325753"
Sep 14 13:38:47.502856 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:47.502825    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ntsnq"
Sep 14 13:38:58.426069 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:58.425971    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:38:59.749274 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:38:59.749243    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:39:07.857268 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:39:07.857236    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:39:14.509455 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:39:14.509423    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:39:16.857347 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:39:16.857312    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:39:32.934219 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:39:32.934184    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:39:35.852341 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:39:35.852311    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:40:08.582023 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:08.581988    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:40:09.783452 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:09.783416    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:40:11.558118 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:11.558083    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:40:18.652721 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:18.652686    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:40:23.029210 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.029171    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"]
Sep 14 13:40:23.032967 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.032949    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:23.036631 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.036601    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 13:40:23.036631 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.036615    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kgb2c\""
Sep 14 13:40:23.036822 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.036609    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 14 13:40:23.036963 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.036945    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 13:40:23.036963 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.036955    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 13:40:23.043484 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.043460    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"]
Sep 14 13:40:23.144505 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.144472    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/062de9e9-2bba-4b59-8fc8-437f255a36ce-maas-api-tls\") pod \"maas-api-577bb5997d-l7cnd\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") " pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:23.144505 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.144506    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/062de9e9-2bba-4b59-8fc8-437f255a36ce-openshift-service-ca\") pod \"maas-api-577bb5997d-l7cnd\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") " pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:23.144746 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.144568    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jrpd\" (UniqueName: \"kubernetes.io/projected/062de9e9-2bba-4b59-8fc8-437f255a36ce-kube-api-access-8jrpd\") pod \"maas-api-577bb5997d-l7cnd\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") " pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:23.144746 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.144616    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/062de9e9-2bba-4b59-8fc8-437f255a36ce-metrics-certs\") pod \"maas-api-577bb5997d-l7cnd\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") " pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:23.245918 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.245886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/062de9e9-2bba-4b59-8fc8-437f255a36ce-maas-api-tls\") pod \"maas-api-577bb5997d-l7cnd\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") " pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:23.245918 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.245915    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/062de9e9-2bba-4b59-8fc8-437f255a36ce-openshift-service-ca\") pod \"maas-api-577bb5997d-l7cnd\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") " pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:23.246152 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.245948    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jrpd\" (UniqueName: \"kubernetes.io/projected/062de9e9-2bba-4b59-8fc8-437f255a36ce-kube-api-access-8jrpd\") pod \"maas-api-577bb5997d-l7cnd\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") " pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:23.246152 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.245980    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/062de9e9-2bba-4b59-8fc8-437f255a36ce-metrics-certs\") pod \"maas-api-577bb5997d-l7cnd\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") " pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:23.246513 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.246486    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/062de9e9-2bba-4b59-8fc8-437f255a36ce-openshift-service-ca\") pod \"maas-api-577bb5997d-l7cnd\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") " pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:23.248327 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.248295    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/062de9e9-2bba-4b59-8fc8-437f255a36ce-maas-api-tls\") pod \"maas-api-577bb5997d-l7cnd\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") " pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:23.248451 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.248429    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/062de9e9-2bba-4b59-8fc8-437f255a36ce-metrics-certs\") pod \"maas-api-577bb5997d-l7cnd\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") " pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:23.253873 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.253854    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jrpd\" (UniqueName: \"kubernetes.io/projected/062de9e9-2bba-4b59-8fc8-437f255a36ce-kube-api-access-8jrpd\") pod \"maas-api-577bb5997d-l7cnd\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") " pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:23.344970 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.344887    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:23.635725 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.635701    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"]
Sep 14 13:40:23.637795 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:40:23.637768    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod062de9e9_2bba_4b59_8fc8_437f255a36ce.slice/crio-68c0c234f47681f36ebd4f3e9e299e1581162d3a6d42739143cd3aa1a605cf62 WatchSource:0}: Error finding container 68c0c234f47681f36ebd4f3e9e299e1581162d3a6d42739143cd3aa1a605cf62: Status 404 returned error can't find the container with id 68c0c234f47681f36ebd4f3e9e299e1581162d3a6d42739143cd3aa1a605cf62
Sep 14 13:40:23.639127 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.639111    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 13:40:23.704868 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.704836    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:40:23.982579 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:23.982486    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd" event={"ID":"062de9e9-2bba-4b59-8fc8-437f255a36ce","Type":"ContainerStarted","Data":"68c0c234f47681f36ebd4f3e9e299e1581162d3a6d42739143cd3aa1a605cf62"}
Sep 14 13:40:24.988430 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:24.988393    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd" event={"ID":"062de9e9-2bba-4b59-8fc8-437f255a36ce","Type":"ContainerStarted","Data":"663602bcdb9b05d381a371cadaff6d3c4785a480fd4c5f46f579df798fe12550"}
Sep 14 13:40:24.988907 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:24.988543    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:25.005316 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:25.005260    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd" podStartSLOduration=1.5361517930000002 podStartE2EDuration="2.005227348s" totalImagesPullingTime="469.075555ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:40:25.005065228 +0000 UTC m=+2518.296399045" watchObservedRunningTime="2026-09-14 13:40:25.005227348 +0000 UTC m=+2518.296561162"
Sep 14 13:40:30.559981 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.559939    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"]
Sep 14 13:40:30.560358 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.560189    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd" podUID="062de9e9-2bba-4b59-8fc8-437f255a36ce" containerName="maas-api" containerID="cri-o://663602bcdb9b05d381a371cadaff6d3c4785a480fd4c5f46f579df798fe12550" gracePeriod=30
Sep 14 13:40:30.564223 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.564198    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:40:30.565248 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.565227    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:30.592784 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.592762    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"]
Sep 14 13:40:30.595884 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.595863    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:30.607377 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.607358    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"]
Sep 14 13:40:30.718115 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.718080    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvmtg\" (UniqueName: \"kubernetes.io/projected/01d7857e-37d9-4d1c-b9db-e4c451a296ab-kube-api-access-mvmtg\") pod \"maas-api-5df7d7c794-mfcrt\" (UID: \"01d7857e-37d9-4d1c-b9db-e4c451a296ab\") " pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:30.718272 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.718132    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01d7857e-37d9-4d1c-b9db-e4c451a296ab-metrics-certs\") pod \"maas-api-5df7d7c794-mfcrt\" (UID: \"01d7857e-37d9-4d1c-b9db-e4c451a296ab\") " pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:30.718272 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.718187    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01d7857e-37d9-4d1c-b9db-e4c451a296ab-maas-api-tls\") pod \"maas-api-5df7d7c794-mfcrt\" (UID: \"01d7857e-37d9-4d1c-b9db-e4c451a296ab\") " pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:30.718357 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.718317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01d7857e-37d9-4d1c-b9db-e4c451a296ab-openshift-service-ca\") pod \"maas-api-5df7d7c794-mfcrt\" (UID: \"01d7857e-37d9-4d1c-b9db-e4c451a296ab\") " pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:30.804821 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.804796    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:30.819425 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.819366    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01d7857e-37d9-4d1c-b9db-e4c451a296ab-maas-api-tls\") pod \"maas-api-5df7d7c794-mfcrt\" (UID: \"01d7857e-37d9-4d1c-b9db-e4c451a296ab\") " pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:30.819556 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.819521    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01d7857e-37d9-4d1c-b9db-e4c451a296ab-openshift-service-ca\") pod \"maas-api-5df7d7c794-mfcrt\" (UID: \"01d7857e-37d9-4d1c-b9db-e4c451a296ab\") " pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:30.819630 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.819615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvmtg\" (UniqueName: \"kubernetes.io/projected/01d7857e-37d9-4d1c-b9db-e4c451a296ab-kube-api-access-mvmtg\") pod \"maas-api-5df7d7c794-mfcrt\" (UID: \"01d7857e-37d9-4d1c-b9db-e4c451a296ab\") " pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:30.819674 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.819641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01d7857e-37d9-4d1c-b9db-e4c451a296ab-metrics-certs\") pod \"maas-api-5df7d7c794-mfcrt\" (UID: \"01d7857e-37d9-4d1c-b9db-e4c451a296ab\") " pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:30.820109 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.820076    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01d7857e-37d9-4d1c-b9db-e4c451a296ab-openshift-service-ca\") pod \"maas-api-5df7d7c794-mfcrt\" (UID: \"01d7857e-37d9-4d1c-b9db-e4c451a296ab\") " pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:30.821786 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.821648    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01d7857e-37d9-4d1c-b9db-e4c451a296ab-maas-api-tls\") pod \"maas-api-5df7d7c794-mfcrt\" (UID: \"01d7857e-37d9-4d1c-b9db-e4c451a296ab\") " pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:30.821916 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.821854    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01d7857e-37d9-4d1c-b9db-e4c451a296ab-metrics-certs\") pod \"maas-api-5df7d7c794-mfcrt\" (UID: \"01d7857e-37d9-4d1c-b9db-e4c451a296ab\") " pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:30.828447 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.828422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvmtg\" (UniqueName: \"kubernetes.io/projected/01d7857e-37d9-4d1c-b9db-e4c451a296ab-kube-api-access-mvmtg\") pod \"maas-api-5df7d7c794-mfcrt\" (UID: \"01d7857e-37d9-4d1c-b9db-e4c451a296ab\") " pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:30.920196 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.920102    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/062de9e9-2bba-4b59-8fc8-437f255a36ce-openshift-service-ca\") pod \"062de9e9-2bba-4b59-8fc8-437f255a36ce\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") "
Sep 14 13:40:30.920371 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.920237    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/062de9e9-2bba-4b59-8fc8-437f255a36ce-maas-api-tls\") pod \"062de9e9-2bba-4b59-8fc8-437f255a36ce\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") "
Sep 14 13:40:30.920371 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.920288    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/062de9e9-2bba-4b59-8fc8-437f255a36ce-metrics-certs\") pod \"062de9e9-2bba-4b59-8fc8-437f255a36ce\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") "
Sep 14 13:40:30.920371 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.920312    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jrpd\" (UniqueName: \"kubernetes.io/projected/062de9e9-2bba-4b59-8fc8-437f255a36ce-kube-api-access-8jrpd\") pod \"062de9e9-2bba-4b59-8fc8-437f255a36ce\" (UID: \"062de9e9-2bba-4b59-8fc8-437f255a36ce\") "
Sep 14 13:40:30.920521 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.920433    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/062de9e9-2bba-4b59-8fc8-437f255a36ce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "062de9e9-2bba-4b59-8fc8-437f255a36ce" (UID: "062de9e9-2bba-4b59-8fc8-437f255a36ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:40:30.920597 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.920580    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/062de9e9-2bba-4b59-8fc8-437f255a36ce-openshift-service-ca\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:40:30.922314 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.922290    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/062de9e9-2bba-4b59-8fc8-437f255a36ce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "062de9e9-2bba-4b59-8fc8-437f255a36ce" (UID: "062de9e9-2bba-4b59-8fc8-437f255a36ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:40:30.922314 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.922306    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/062de9e9-2bba-4b59-8fc8-437f255a36ce-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "062de9e9-2bba-4b59-8fc8-437f255a36ce" (UID: "062de9e9-2bba-4b59-8fc8-437f255a36ce"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:40:30.922428 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.922343    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/062de9e9-2bba-4b59-8fc8-437f255a36ce-kube-api-access-8jrpd" (OuterVolumeSpecName: "kube-api-access-8jrpd") pod "062de9e9-2bba-4b59-8fc8-437f255a36ce" (UID: "062de9e9-2bba-4b59-8fc8-437f255a36ce"). InnerVolumeSpecName "kube-api-access-8jrpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:40:30.937539 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:30.937513    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:31.021932 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:31.021901    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/062de9e9-2bba-4b59-8fc8-437f255a36ce-maas-api-tls\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:40:31.021932 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:31.021935    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/062de9e9-2bba-4b59-8fc8-437f255a36ce-metrics-certs\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:40:31.022145 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:31.021949    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jrpd\" (UniqueName: \"kubernetes.io/projected/062de9e9-2bba-4b59-8fc8-437f255a36ce-kube-api-access-8jrpd\") on node \"ip-10-0-135-35.ec2.internal\" DevicePath \"\""
Sep 14 13:40:31.022879 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:31.022853    2581 generic.go:356] "Generic (PLEG): container finished" podID="062de9e9-2bba-4b59-8fc8-437f255a36ce" containerID="663602bcdb9b05d381a371cadaff6d3c4785a480fd4c5f46f579df798fe12550" exitCode=0
Sep 14 13:40:31.022976 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:31.022898    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"
Sep 14 13:40:31.022976 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:31.022940    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd" event={"ID":"062de9e9-2bba-4b59-8fc8-437f255a36ce","Type":"ContainerDied","Data":"663602bcdb9b05d381a371cadaff6d3c4785a480fd4c5f46f579df798fe12550"}
Sep 14 13:40:31.023090 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:31.023001    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd" event={"ID":"062de9e9-2bba-4b59-8fc8-437f255a36ce","Type":"ContainerDied","Data":"68c0c234f47681f36ebd4f3e9e299e1581162d3a6d42739143cd3aa1a605cf62"}
Sep 14 13:40:31.023090 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:31.023034    2581 scope.go:117] "RemoveContainer" containerID="663602bcdb9b05d381a371cadaff6d3c4785a480fd4c5f46f579df798fe12550"
Sep 14 13:40:31.031847 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:31.031814    2581 scope.go:117] "RemoveContainer" containerID="663602bcdb9b05d381a371cadaff6d3c4785a480fd4c5f46f579df798fe12550"
Sep 14 13:40:31.032117 ip-10-0-135-35 kubenswrapper[2581]: E0914 13:40:31.032093    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"663602bcdb9b05d381a371cadaff6d3c4785a480fd4c5f46f579df798fe12550\": container with ID starting with 663602bcdb9b05d381a371cadaff6d3c4785a480fd4c5f46f579df798fe12550 not found: ID does not exist" containerID="663602bcdb9b05d381a371cadaff6d3c4785a480fd4c5f46f579df798fe12550"
Sep 14 13:40:31.032183 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:31.032125    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"663602bcdb9b05d381a371cadaff6d3c4785a480fd4c5f46f579df798fe12550"} err="failed to get container status \"663602bcdb9b05d381a371cadaff6d3c4785a480fd4c5f46f579df798fe12550\": rpc error: code = NotFound desc = could not find container \"663602bcdb9b05d381a371cadaff6d3c4785a480fd4c5f46f579df798fe12550\": container with ID starting with 663602bcdb9b05d381a371cadaff6d3c4785a480fd4c5f46f579df798fe12550 not found: ID does not exist"
Sep 14 13:40:31.043979 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:31.043959    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"]
Sep 14 13:40:31.047626 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:31.047606    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-577bb5997d-l7cnd"]
Sep 14 13:40:31.206503 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:31.206478    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"]
Sep 14 13:40:31.208812 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:40:31.208785    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01d7857e_37d9_4d1c_b9db_e4c451a296ab.slice/crio-b924888d4fa840c5f273f0fc124bdefdaf8e80796ccad62869c6ceb7ddc472b3 WatchSource:0}: Error finding container b924888d4fa840c5f273f0fc124bdefdaf8e80796ccad62869c6ceb7ddc472b3: Status 404 returned error can't find the container with id b924888d4fa840c5f273f0fc124bdefdaf8e80796ccad62869c6ceb7ddc472b3
Sep 14 13:40:31.245392 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:31.245365    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="062de9e9-2bba-4b59-8fc8-437f255a36ce" path="/var/lib/kubelet/pods/062de9e9-2bba-4b59-8fc8-437f255a36ce/volumes"
Sep 14 13:40:32.029753 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:32.029715    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt" event={"ID":"01d7857e-37d9-4d1c-b9db-e4c451a296ab","Type":"ContainerStarted","Data":"c1c428424abb9bb4e7b77fce06a1093e611bea1690245b3787fa9e0b40d900dd"}
Sep 14 13:40:32.030147 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:32.029776    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt" event={"ID":"01d7857e-37d9-4d1c-b9db-e4c451a296ab","Type":"ContainerStarted","Data":"b924888d4fa840c5f273f0fc124bdefdaf8e80796ccad62869c6ceb7ddc472b3"}
Sep 14 13:40:32.030147 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:32.029927    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:32.050608 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:32.050553    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt" podStartSLOduration=1.675034652 podStartE2EDuration="2.050519533s" totalImagesPullingTime="375.484881ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:40:32.047020336 +0000 UTC m=+2525.338354152" watchObservedRunningTime="2026-09-14 13:40:32.050519533 +0000 UTC m=+2525.341853349"
Sep 14 13:40:37.261897 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:37.261867    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:40:38.039483 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:38.039457    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5df7d7c794-mfcrt"
Sep 14 13:40:43.047158 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:43.047119    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:40:49.850879 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:49.850844    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-pjqrn"]
Sep 14 13:40:58.764247 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:58.764215    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-697cf97c89-mwdgg_527f4585-b998-46ec-a189-ff50c371c3a9/copy-manifests/0.log"
Sep 14 13:40:58.770509 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:58.770486    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-697cf97c89-mwdgg_527f4585-b998-46ec-a189-ff50c371c3a9/manager/0.log"
Sep 14 13:40:59.111658 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:59.111630    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5df7d7c794-mfcrt_01d7857e-37d9-4d1c-b9db-e4c451a296ab/maas-api/0.log"
Sep 14 13:40:59.330974 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:40:59.330945    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29823195-65drz_cd18b195-fbf1-4ed8-9dd3-c5c5f68a6e07/cleanup/0.log"
Sep 14 13:41:00.750348 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:00.750318    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-ntsnq_f1b864c0-dcc0-4baf-a529-e0f94cdef80d/manager/0.log"
Sep 14 13:41:00.867683 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:00.867655    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-pjqrn_46dc4cee-e2f4-4854-b6c9-264795d5c70f/limitador/0.log"
Sep 14 13:41:01.666797 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:01.666767    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f6f8d7966-tzrvk_662db2d6-5df3-499a-9052-8be6ec86e6e3/kube-auth-proxy/0.log"
Sep 14 13:41:02.772718 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:02.772687    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-mj64s_7baae410-2e01-4d0f-aa09-e85fdc179263/main/0.log"
Sep 14 13:41:03.098902 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:03.098822    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-9mhrl_4708d8c6-a33e-459f-b87c-616d1525de82/main/0.log"
Sep 14 13:41:08.535466 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:08.535430    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:41:08.594391 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:08.594359    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log" limit=1024
Sep 14 13:41:11.313698 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:11.313660    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dszvl_db77a7f7-b9e7-47f8-a7d0-524847920a6b/global-pull-secret-syncer/0.log"
Sep 14 13:41:11.398506 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:11.398477    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9462f_48b049b1-8dce-45eb-b6e0-2ce78ec5b7b4/konnectivity-agent/0.log"
Sep 14 13:41:11.512149 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:11.512118    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-35.ec2.internal_1db45f87cf9fbe960d92124e3bae8a9e/haproxy/0.log"
Sep 14 13:41:16.252320 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:16.252290    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-ntsnq_f1b864c0-dcc0-4baf-a529-e0f94cdef80d/manager/0.log"
Sep 14 13:41:16.272421 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:16.272395    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-pjqrn_46dc4cee-e2f4-4854-b6c9-264795d5c70f/limitador/0.log"
Sep 14 13:41:17.939297 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:17.939269    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-95td8_d2750404-ec81-431c-b230-2dd1fe53560f/kube-state-metrics/0.log"
Sep 14 13:41:17.958663 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:17.958643    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-95td8_d2750404-ec81-431c-b230-2dd1fe53560f/kube-rbac-proxy-main/0.log"
Sep 14 13:41:17.976425 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:17.976400    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-95td8_d2750404-ec81-431c-b230-2dd1fe53560f/kube-rbac-proxy-self/0.log"
Sep 14 13:41:18.085586 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.085557    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-62zc6_b443dc84-d2e7-4260-8076-8c0a90d9f275/node-exporter/0.log"
Sep 14 13:41:18.117378 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.117333    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-62zc6_b443dc84-d2e7-4260-8076-8c0a90d9f275/kube-rbac-proxy/0.log"
Sep 14 13:41:18.142753 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.142732    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-62zc6_b443dc84-d2e7-4260-8076-8c0a90d9f275/init-textfile/0.log"
Sep 14 13:41:18.361876 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.361849    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-8gtwj_c1c76a80-a05b-4913-85e3-d85d8174a8c8/kube-rbac-proxy-main/0.log"
Sep 14 13:41:18.379471 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.379449    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-8gtwj_c1c76a80-a05b-4913-85e3-d85d8174a8c8/kube-rbac-proxy-self/0.log"
Sep 14 13:41:18.397124 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.397102    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-8gtwj_c1c76a80-a05b-4913-85e3-d85d8174a8c8/openshift-state-metrics/0.log"
Sep 14 13:41:18.428123 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.428101    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e64b2c7b-8c6e-48ff-9b8b-15a813581b97/prometheus/0.log"
Sep 14 13:41:18.449506 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.449487    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e64b2c7b-8c6e-48ff-9b8b-15a813581b97/config-reloader/0.log"
Sep 14 13:41:18.473317 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.473298    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e64b2c7b-8c6e-48ff-9b8b-15a813581b97/thanos-sidecar/0.log"
Sep 14 13:41:18.486360 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.486331    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e64b2c7b-8c6e-48ff-9b8b-15a813581b97/kube-rbac-proxy-web/0.log"
Sep 14 13:41:18.506944 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.506890    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e64b2c7b-8c6e-48ff-9b8b-15a813581b97/kube-rbac-proxy/0.log"
Sep 14 13:41:18.528788 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.528770    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e64b2c7b-8c6e-48ff-9b8b-15a813581b97/kube-rbac-proxy-thanos/0.log"
Sep 14 13:41:18.557013 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.556993    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e64b2c7b-8c6e-48ff-9b8b-15a813581b97/init-config-reloader/0.log"
Sep 14 13:41:18.586119 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.586102    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-dgrgw_3c798898-5ae9-48a9-a02d-85cbeaa8528b/prometheus-operator/0.log"
Sep 14 13:41:18.603355 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.603337    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-dgrgw_3c798898-5ae9-48a9-a02d-85cbeaa8528b/kube-rbac-proxy/0.log"
Sep 14 13:41:18.625404 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.625384    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-ccxkp_4fc88f14-34e1-4dd9-beaa-7addd2b64afb/prometheus-operator-admission-webhook/0.log"
Sep 14 13:41:18.733127 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.733105    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dcd647455-8hnmd_63dc1702-6042-41a9-96d0-1e623b94ff18/thanos-query/0.log"
Sep 14 13:41:18.753147 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.753123    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dcd647455-8hnmd_63dc1702-6042-41a9-96d0-1e623b94ff18/kube-rbac-proxy-web/0.log"
Sep 14 13:41:18.771075 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.771021    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dcd647455-8hnmd_63dc1702-6042-41a9-96d0-1e623b94ff18/kube-rbac-proxy/0.log"
Sep 14 13:41:18.789690 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.789666    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dcd647455-8hnmd_63dc1702-6042-41a9-96d0-1e623b94ff18/prom-label-proxy/0.log"
Sep 14 13:41:18.809758 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.809738    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dcd647455-8hnmd_63dc1702-6042-41a9-96d0-1e623b94ff18/kube-rbac-proxy-rules/0.log"
Sep 14 13:41:18.826871 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:18.826851    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dcd647455-8hnmd_63dc1702-6042-41a9-96d0-1e623b94ff18/kube-rbac-proxy-metrics/0.log"
Sep 14 13:41:21.432465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.432430    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"]
Sep 14 13:41:21.432931 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.432797    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="062de9e9-2bba-4b59-8fc8-437f255a36ce" containerName="maas-api"
Sep 14 13:41:21.432931 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.432816    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="062de9e9-2bba-4b59-8fc8-437f255a36ce" containerName="maas-api"
Sep 14 13:41:21.432931 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.432927    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="062de9e9-2bba-4b59-8fc8-437f255a36ce" containerName="maas-api"
Sep 14 13:41:21.436089 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.436072    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.438801 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.438783    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2dnwm\"/\"default-dockercfg-5mfhs\""
Sep 14 13:41:21.438916 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.438832    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2dnwm\"/\"openshift-service-ca.crt\""
Sep 14 13:41:21.438916 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.438857    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2dnwm\"/\"kube-root-ca.crt\""
Sep 14 13:41:21.444385 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.444363    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"]
Sep 14 13:41:21.512596 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.512571    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-swpmp_229dfed7-2ade-4f61-8c09-333b42988051/download-server/0.log"
Sep 14 13:41:21.555344 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.555319    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5f3971e-e59a-43b2-8ab8-e693fb03b855-sys\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.555465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.555355    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88ds8\" (UniqueName: \"kubernetes.io/projected/d5f3971e-e59a-43b2-8ab8-e693fb03b855-kube-api-access-88ds8\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.555465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.555391    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5f3971e-e59a-43b2-8ab8-e693fb03b855-podres\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.555465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.555416    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5f3971e-e59a-43b2-8ab8-e693fb03b855-lib-modules\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.555465 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.555434    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5f3971e-e59a-43b2-8ab8-e693fb03b855-proc\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.656372 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.656342    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5f3971e-e59a-43b2-8ab8-e693fb03b855-podres\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.656492 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.656385    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5f3971e-e59a-43b2-8ab8-e693fb03b855-lib-modules\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.656492 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.656410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5f3971e-e59a-43b2-8ab8-e693fb03b855-proc\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.656492 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.656462    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5f3971e-e59a-43b2-8ab8-e693fb03b855-sys\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.656492 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.656483    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5f3971e-e59a-43b2-8ab8-e693fb03b855-proc\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.656692 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.656500    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88ds8\" (UniqueName: \"kubernetes.io/projected/d5f3971e-e59a-43b2-8ab8-e693fb03b855-kube-api-access-88ds8\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.656692 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.656503    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5f3971e-e59a-43b2-8ab8-e693fb03b855-podres\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.656692 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.656541    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5f3971e-e59a-43b2-8ab8-e693fb03b855-lib-modules\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.656692 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.656558    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5f3971e-e59a-43b2-8ab8-e693fb03b855-sys\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.665366 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.665339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88ds8\" (UniqueName: \"kubernetes.io/projected/d5f3971e-e59a-43b2-8ab8-e693fb03b855-kube-api-access-88ds8\") pod \"perf-node-gather-daemonset-vb5rf\" (UID: \"d5f3971e-e59a-43b2-8ab8-e693fb03b855\") " pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:21.746084 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:21.746018    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:22.015040 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:22.015017    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"]
Sep 14 13:41:22.017127 ip-10-0-135-35 kubenswrapper[2581]: W0914 13:41:22.017101    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd5f3971e_e59a_43b2_8ab8_e693fb03b855.slice/crio-8f8501e637a61b724f277e9bfc6c4b14c174923965f13fc85dc4c7f5bec4402d WatchSource:0}: Error finding container 8f8501e637a61b724f277e9bfc6c4b14c174923965f13fc85dc4c7f5bec4402d: Status 404 returned error can't find the container with id 8f8501e637a61b724f277e9bfc6c4b14c174923965f13fc85dc4c7f5bec4402d
Sep 14 13:41:22.257740 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:22.257703    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf" event={"ID":"d5f3971e-e59a-43b2-8ab8-e693fb03b855","Type":"ContainerStarted","Data":"f8ff4f9a265b7fb417eb25164cd9d3397159152bd70e67ca226ba4339808e91f"}
Sep 14 13:41:22.257926 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:22.257766    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:22.257926 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:22.257790    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf" event={"ID":"d5f3971e-e59a-43b2-8ab8-e693fb03b855","Type":"ContainerStarted","Data":"8f8501e637a61b724f277e9bfc6c4b14c174923965f13fc85dc4c7f5bec4402d"}
Sep 14 13:41:22.274742 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:22.274655    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf" podStartSLOduration=1.274641136 podStartE2EDuration="1.274641136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 13:41:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:41:22.274512673 +0000 UTC m=+2575.565846489" watchObservedRunningTime="2026-09-14 13:41:22.274641136 +0000 UTC m=+2575.565974952"
Sep 14 13:41:23.339231 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:23.339191    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g2bqm_c32a3bdc-bd26-4891-843d-0808ffa7ec6d/dns/0.log"
Sep 14 13:41:23.358262 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:23.358242    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g2bqm_c32a3bdc-bd26-4891-843d-0808ffa7ec6d/kube-rbac-proxy/0.log"
Sep 14 13:41:23.515348 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:23.515322    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vftv8_4cc3d786-1998-4dd8-b68d-739249a7ec18/dns-node-resolver/0.log"
Sep 14 13:41:24.073709 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:24.073683    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7fcb6fd65f-jkfsm_c0970c76-9964-4645-b9a9-92aa45a58fc0/registry/0.log"
Sep 14 13:41:24.135310 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:24.135280    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t797n_8e6b0822-d820-47f4-a652-f56f2f95707e/node-ca/0.log"
Sep 14 13:41:25.912489 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:25.912460    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f6f8d7966-tzrvk_662db2d6-5df3-499a-9052-8be6ec86e6e3/kube-auth-proxy/0.log"
Sep 14 13:41:26.932058 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:26.932029    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xf7sp_9cec680f-1687-4f62-af5d-43a9c8f6f630/serve-healthcheck-canary/0.log"
Sep 14 13:41:27.537471 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:27.537438    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bnpj9_c20fe060-6c64-4048-ac0c-bf214312d5ad/kube-rbac-proxy/0.log"
Sep 14 13:41:27.554001 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:27.553976    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bnpj9_c20fe060-6c64-4048-ac0c-bf214312d5ad/exporter/0.log"
Sep 14 13:41:27.570618 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:27.570593    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bnpj9_c20fe060-6c64-4048-ac0c-bf214312d5ad/extractor/0.log"
Sep 14 13:41:28.271248 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:28.271219    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2dnwm/perf-node-gather-daemonset-vb5rf"
Sep 14 13:41:30.338165 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:30.338138    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-697cf97c89-mwdgg_527f4585-b998-46ec-a189-ff50c371c3a9/manager/0.log"
Sep 14 13:41:30.355042 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:30.355023    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-697cf97c89-mwdgg_527f4585-b998-46ec-a189-ff50c371c3a9/copy-manifests/0.log"
Sep 14 13:41:31.661028 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:31.661001    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6d5f86f746-p8fmq_360df0fb-ef47-4edd-83fb-24abdf61f92d/manager/0.log"
Sep 14 13:41:31.969700 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:31.969673    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f6f8d7966-tzrvk_662db2d6-5df3-499a-9052-8be6ec86e6e3/kube-auth-proxy/0.log"
Sep 14 13:41:34.536926 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:34.536850    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f6f8d7966-tzrvk_662db2d6-5df3-499a-9052-8be6ec86e6e3/kube-auth-proxy/0.log"
Sep 14 13:41:36.737849 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:36.737824    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-r8tc5_f9e5b8f0-56f7-4bf9-bfe2-e420ec105e4c/migrator/0.log"
Sep 14 13:41:36.754876 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:36.754853    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-r8tc5_f9e5b8f0-56f7-4bf9-bfe2-e420ec105e4c/graceful-termination/0.log"
Sep 14 13:41:38.467471 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:38.467443    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zdlhr_b59f7c6d-e692-4bb7-baca-3b4e6c932134/kube-multus-additional-cni-plugins/0.log"
Sep 14 13:41:38.488260 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:38.488236    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zdlhr_b59f7c6d-e692-4bb7-baca-3b4e6c932134/egress-router-binary-copy/0.log"
Sep 14 13:41:38.508093 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:38.508043    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zdlhr_b59f7c6d-e692-4bb7-baca-3b4e6c932134/cni-plugins/0.log"
Sep 14 13:41:38.528285 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:38.528270    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zdlhr_b59f7c6d-e692-4bb7-baca-3b4e6c932134/bond-cni-plugin/0.log"
Sep 14 13:41:38.550046 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:38.550028    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zdlhr_b59f7c6d-e692-4bb7-baca-3b4e6c932134/routeoverride-cni/0.log"
Sep 14 13:41:38.571105 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:38.571084    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zdlhr_b59f7c6d-e692-4bb7-baca-3b4e6c932134/whereabouts-cni-bincopy/0.log"
Sep 14 13:41:38.594002 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:38.593986    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zdlhr_b59f7c6d-e692-4bb7-baca-3b4e6c932134/whereabouts-cni/0.log"
Sep 14 13:41:38.753357 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:38.753327    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-sfhjz_1346f708-a2ee-4552-8a2c-1de1e10b4277/kube-multus/0.log"
Sep 14 13:41:38.771460 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:38.771399    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-49fdc_f05d48a3-962c-42c0-aba5-1cd6441c74d9/network-metrics-daemon/0.log"
Sep 14 13:41:38.790041 ip-10-0-135-35 kubenswrapper[2581]: I0914 13:41:38.790020    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-49fdc_f05d48a3-962c-42c0-aba5-1cd6441c74d9/kube-rbac-proxy/0.log"