Sep 07 08:56:55.365407 ip-10-0-141-81 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 07 08:56:55.365420 ip-10-0-141-81 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 07 08:56:55.365429 ip-10-0-141-81 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 07 08:56:55.365647 ip-10-0-141-81 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 07 08:57:04.824742 ip-10-0-141-81 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Sep 07 08:57:04.824755 ip-10-0-141-81 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 343c8c7c2aca4433b4b5aad90eb58fc9 --
Sep 07 08:58:36.922963 ip-10-0-141-81 systemd[1]: Starting Kubernetes Kubelet...
Sep 07 08:58:37.341487 ip-10-0-141-81 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 08:58:37.341487 ip-10-0-141-81 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 07 08:58:37.341487 ip-10-0-141-81 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 08:58:37.341487 ip-10-0-141-81 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 07 08:58:37.341487 ip-10-0-141-81 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 08:58:37.342148 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.341542    2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 07 08:58:37.349999 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.349974    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 08:58:37.349999 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.349993    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 08:58:37.349999 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.349996    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 08:58:37.349999 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.349999    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 08:58:37.349999 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350003    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 08:58:37.349999 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350006    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350010    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350014    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350017    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350019    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350022    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350025    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350027    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350030    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350032    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350035    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350037    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350040    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350042    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350045    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350048    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350050    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350053    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350055    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350058    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 08:58:37.350204 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350061    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350064    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350066    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350069    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350072    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350074    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350077    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350080    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350082    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350085    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350088    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350090    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350092    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350095    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350098    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350100    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350103    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350106    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350108    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 08:58:37.350692 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350112    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350116    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350118    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350121    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350123    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350126    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350130    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350132    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350135    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350137    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350140    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350143    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350145    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350148    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350151    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350153    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350156    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350160    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350164    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350167    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 08:58:37.351190 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350169    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350172    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350176    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350178    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350181    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350184    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350186    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350189    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350192    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350195    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350197    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350200    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350203    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350206    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350209    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350212    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350216    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350219    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350222    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 08:58:37.351766 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350225    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350228    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350230    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350233    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350235    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350238    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350241    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350243    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350246    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350249    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350252    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350255    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350258    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350260    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350263    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350265    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350966    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350974    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350977    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350980    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 08:58:37.352232 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350985    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350988    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350993    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350996    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.350999    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351002    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351005    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351007    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351010    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351012    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351015    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351017    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351021    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351025    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351028    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351031    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351033    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351038    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351040    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 08:58:37.352719 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351043    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351045    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351047    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351050    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351053    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351055    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351058    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351060    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351063    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351065    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351068    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351070    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351073    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351077    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351080    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351082    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351085    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351087    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351090    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351092    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 08:58:37.353187 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351095    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351098    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351101    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351103    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351106    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351108    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351111    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351114    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351116    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351118    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351121    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351123    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351126    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351128    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351130    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351133    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351135    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351138    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351140    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351143    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 08:58:37.353678 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351145    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351148    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351150    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351153    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351155    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351159    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351162    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351166    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351169    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351172    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351175    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351177    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351180    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351183    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351186    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351189    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351191    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351194    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351198    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351202    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 08:58:37.354183 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351205    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351208    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351210    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351213    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351215    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351218    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351220    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351223    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351225    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351227    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351230    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351232    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351235    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351237    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351239    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351242    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351583    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351589    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351592    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351595    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 08:58:37.354669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351598    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351600    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351603    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351606    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351608    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351611    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351613    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351616    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351618    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351621    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351624    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351626    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351629    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351632    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351635    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351637    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351640    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351642    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351645    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351647    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 08:58:37.355176 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351650    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351652    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351656    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351660    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351662    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351665    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351668    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351670    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351673    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351676    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351679    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351682    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351686    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351689    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351692    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351695    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351698    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351701    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351704    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 08:58:37.355652 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351706    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351728    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351733    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351736    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351738    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351740    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351745    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351749    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351751    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351754    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351756    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351759    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351761    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351764    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351766    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351769    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351772    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351774    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351777    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351779    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 08:58:37.356148 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351782    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351784    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351788    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351795    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351797    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351800    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351802    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351804    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351807    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351810    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351812    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351815    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351817    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351819    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351822    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351824    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351827    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351829    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351832    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351838    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 08:58:37.356619 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351842    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351845    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351847    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351850    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351852    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351854    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351857    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351859    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351862    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351864    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351867    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351869    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351872    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351874    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351878    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.351882    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.351958    2574 flags.go:64] FLAG: --address="0.0.0.0"
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352002    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352009    2574 flags.go:64] FLAG: --anonymous-auth="true"
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352013    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352017    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 07 08:58:37.357131 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352020    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352024    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352028    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352031    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352034    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352037    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352041    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352044    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352047    2574 flags.go:64] FLAG: --cgroup-root=""
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352050    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352053    2574 flags.go:64] FLAG: --client-ca-file=""
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352056    2574 flags.go:64] FLAG: --cloud-provider="external"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352059    2574 flags.go:64] FLAG: --cluster-dns="[]"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352067    2574 flags.go:64] FLAG: --cluster-domain=""
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352070    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352073    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352076    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352079    2574 flags.go:64] FLAG: --container-log-max-files="5"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352087    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352089    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352092    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352095    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352098    2574 flags.go:64] FLAG: --contention-profiling="false"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352101    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 07 08:58:37.357623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352104    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352108    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352113    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352117    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352120    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352122    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352125    2574 flags.go:64] FLAG: --enable-load-reader="false"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352128    2574 flags.go:64] FLAG: --enable-server="true"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352131    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352135    2574 flags.go:64] FLAG: --event-burst="100"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352138    2574 flags.go:64] FLAG: --event-qps="50"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352141    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352143    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352146    2574 flags.go:64] FLAG: --eviction-hard=""
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352150    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352153    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352155    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352158    2574 flags.go:64] FLAG: --eviction-soft=""
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352161    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352164    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352167    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352170    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352173    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352176    2574 flags.go:64] FLAG: --fail-swap-on="true"
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352178    2574 flags.go:64] FLAG: --feature-gates=""
Sep 07 08:58:37.358197 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352182    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352185    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352188    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352191    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352194    2574 flags.go:64] FLAG: --healthz-port="10248"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352197    2574 flags.go:64] FLAG: --help="false"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352199    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352202    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352206    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352210    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352214    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352217    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352220    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352223    2574 flags.go:64] FLAG: --image-service-endpoint=""
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352225    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352228    2574 flags.go:64] FLAG: --kube-api-burst="100"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352231    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352234    2574 flags.go:64] FLAG: --kube-api-qps="50"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352237    2574 flags.go:64] FLAG: --kube-reserved=""
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352240    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352243    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352247    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352249    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352252    2574 flags.go:64] FLAG: --lock-file=""
Sep 07 08:58:37.358888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352255    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352258    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352260    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352266    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352269    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352271    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352274    2574 flags.go:64] FLAG: --logging-format="text"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352277    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352280    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352282    2574 flags.go:64] FLAG: --manifest-url=""
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352285    2574 flags.go:64] FLAG: --manifest-url-header=""
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352289    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352292    2574 flags.go:64] FLAG: --max-open-files="1000000"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352296    2574 flags.go:64] FLAG: --max-pods="110"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352299    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352301    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352306    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352310    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352313    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352316    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352318    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352325    2574 flags.go:64] FLAG: --node-status-max-images="50"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352328    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352331    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 07 08:58:37.359473 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352334    2574 flags.go:64] FLAG: --pod-cidr=""
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352337    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352341    2574 flags.go:64] FLAG: --pod-manifest-path=""
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352344    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352347    2574 flags.go:64] FLAG: --pods-per-core="0"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352350    2574 flags.go:64] FLAG: --port="10250"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352353    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352356    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0afc4d742da34c2e4"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352359    2574 flags.go:64] FLAG: --qos-reserved=""
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352361    2574 flags.go:64] FLAG: --read-only-port="10255"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352364    2574 flags.go:64] FLAG: --register-node="true"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352367    2574 flags.go:64] FLAG: --register-with-taints=""
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352370    2574 flags.go:64] FLAG: --registry-burst="10"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352374    2574 flags.go:64] FLAG: --registry-qps="5"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352376    2574 flags.go:64] FLAG: --reserved-cpus=""
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352379    2574 flags.go:64] FLAG: --reserved-memory=""
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352383    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352386    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352389    2574 flags.go:64] FLAG: --rotate-certificates="false"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352391    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352394    2574 flags.go:64] FLAG: --runonce="false"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352397    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352399    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352402    2574 flags.go:64] FLAG: --seccomp-default="false"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352405    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 07 08:58:37.360071 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352410    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352413    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352416    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352419    2574 flags.go:64] FLAG: --storage-driver-password="root"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352421    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352424    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352427    2574 flags.go:64] FLAG: --storage-driver-user="root"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352430    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352432    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352435    2574 flags.go:64] FLAG: --system-cgroups=""
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352438    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352443    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352446    2574 flags.go:64] FLAG: --tls-cert-file=""
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352449    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352452    2574 flags.go:64] FLAG: --tls-min-version=""
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352454    2574 flags.go:64] FLAG: --tls-private-key-file=""
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352457    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352460    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352462    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352465    2574 flags.go:64] FLAG: --v="2"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352469    2574 flags.go:64] FLAG: --version="false"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352473    2574 flags.go:64] FLAG: --vmodule=""
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352476    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.352480    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352582    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 08:58:37.360669 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352586    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352589    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352592    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352594    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352598    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352602    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352605    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352609    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352612    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352615    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352618    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352620    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352623    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352625    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352628    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352630    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352633    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352635    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352638    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352641    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 08:58:37.361288 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352643    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352645    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352648    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352650    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352653    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352655    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352657    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352660    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352663    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352666    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352668    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352670    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352673    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352675    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352678    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352680    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352682    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352685    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352687    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352691    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 08:58:37.361792 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352694    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352696    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352699    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352701    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352703    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352723    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352728    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352731    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352733    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352736    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352738    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352741    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352743    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352745    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352748    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352750    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352754    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352756    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352759    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352761    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 08:58:37.362311 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352764    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352766    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352771    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352774    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352776    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352779    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352781    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352783    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352786    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352788    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352790    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352795    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352798    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352800    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352803    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352805    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352807    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352810    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352813    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352815    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 08:58:37.362824 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352817    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352820    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352822    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352824    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352827    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352829    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352831    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352834    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352838    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352840    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352843    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352845    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352847    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352849    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352852    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352855    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352859    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 08:58:37.363306 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.352862    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.353628    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.361445    2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.361462    2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.362437    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.362445    2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362509    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362514    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362518    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362521    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362524    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362527    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362530    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362533    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362536    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 08:58:37.363790 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362539    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362542    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362546    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362548    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362551    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362554    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362557    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362560    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362563    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362566    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362570    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362573    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362576    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362579    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362581    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362584    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362587    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362590    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362592    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362595    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 08:58:37.364181 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362599    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362601    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362604    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362607    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362609    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362612    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362615    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362617    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362620    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362622    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362625    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362628    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362630    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362633    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362635    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362638    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362640    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362643    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362645    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362648    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 08:58:37.364657 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362651    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362654    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362656    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362658    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362661    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362664    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362666    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362669    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362671    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362674    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362676    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362679    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362682    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362685    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362688    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362691    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362693    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362696    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362699    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362701    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 08:58:37.365141 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362704    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362724    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362728    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362731    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362733    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362736    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362739    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362742    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362744    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362747    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362749    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362752    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362756    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362760    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362764    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362767    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362770    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362772    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362775    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 08:58:37.365660 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362778    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362781    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362785    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362788    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362791    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362794    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362797    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362800    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362803    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362806    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362808    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.362813    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362917    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362921    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362924    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 08:58:37.366133 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362927    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362930    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362933    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362935    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362938    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362940    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362943    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362946    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362948    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362951    2574 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362955    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362958    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362961    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362965    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362967    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362970    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362972    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362975    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362978    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 08:58:37.366497 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362980    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362983    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362986    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362988    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362991    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362994    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362997    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.362999    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363001    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363004    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363006    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363008    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363011    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363013    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363016    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363018    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363021    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363023    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363025    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 08:58:37.366967 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363028    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363030    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363032    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363035    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363037    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363039    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363042    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363044    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363046    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363049    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363051    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363054    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363056    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363058    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363061    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363064    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363068    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363071    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363074    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363077    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 08:58:37.367422 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363080    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363083    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363086    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363089    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363092    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363095    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363098    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363100    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363103    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363106    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363108    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363111    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363114    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363116    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363119    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363121    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363124    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363127    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363129    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 08:58:37.367911 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363131    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363134    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363136    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363139    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363141    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363144    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363146    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363149    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363151    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363153    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363156    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363159    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363161    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363163    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363166    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363169    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363171    2574 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363174    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 08:58:37.368360 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:37.363176    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 08:58:37.368874 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.363181    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 08:58:37.368874 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.363272    2574 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 07 08:58:37.368874 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.365534    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 07 08:58:37.368874 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.366621    2574 server.go:1019] "Starting client certificate rotation"
Sep 07 08:58:37.368874 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.366736    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 08:58:37.368874 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.366765    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 08:58:37.369513 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.369501    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 08:58:37.371917 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.370955    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 08:58:37.383148 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.383131    2574 log.go:25] "Validated CRI v1 runtime API"
Sep 07 08:58:37.387825 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.387808    2574 log.go:25] "Validated CRI v1 image API"
Sep 07 08:58:37.388668 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.388651    2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 07 08:58:37.393015 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.392990    2574 fs.go:135] Filesystem UUIDs: map[0f294280-4b99-4753-8791-dbd150fb5d4d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 94e6eda0-7484-4a65-b4a1-cc1d927f3c23:/dev/nvme0n1p3]
Sep 07 08:58:37.393072 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.393013    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 07 08:58:37.397756 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.397734    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 08:58:37.398390 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.398291    2574 manager.go:216] Machine: {Timestamp:2026-09-07 08:58:37.396731263 +0000 UTC m=+0.360647497 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3057629 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec211101a426dfcd74e5bb796c34fbfb SystemUUID:ec211101-a426-dfcd-74e5-bb796c34fbfb BootID:343c8c7c-2aca-4433-b4b5-aad90eb58fc9 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7a:39:bc:9b:91 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7a:39:bc:9b:91 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:fb:d0:7d:ed:0b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 07 08:58:37.398390 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.398391    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 07 08:58:37.398518 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.398503    2574 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 07 08:58:37.402442 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.402424    2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 07 08:58:37.402588 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.402445    2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-81.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 07 08:58:37.403251 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.403241    2574 topology_manager.go:138] "Creating topology manager with none policy"
Sep 07 08:58:37.403283 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.403254    2574 container_manager_linux.go:309] "Creating device plugin manager"
Sep 07 08:58:37.403283 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.403261    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 08:58:37.403283 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.403275    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 08:58:37.405032 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.405022    2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 07 08:58:37.409411 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.409400    2574 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 08:58:37.409526 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.409517    2574 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 07 08:58:37.409559 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.409544    2574 kubelet.go:486] "Attempting to sync node with API server"
Sep 07 08:58:37.409589 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.409560    2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 07 08:58:37.409589 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.409574    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 07 08:58:37.409589 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.409583    2574 kubelet.go:398] "Adding apiserver pod source"
Sep 07 08:58:37.409675 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.409591    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 07 08:58:37.409914 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.409902    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 08:58:37.409972 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.409923    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 08:58:37.412406 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.412389    2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 07 08:58:37.415149 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.415136    2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 07 08:58:37.415192 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.415166    2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 07 08:58:37.415321 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.415311    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 07 08:58:37.415350 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.415327    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 07 08:58:37.415388 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.415361    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 07 08:58:37.415388 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.415372    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 07 08:58:37.415388 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.415383    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 07 08:58:37.415469 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.415391    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 07 08:58:37.416020 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.416011    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 07 08:58:37.416050 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.416020    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 07 08:58:37.416050 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.416031    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 07 08:58:37.416050 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.416036    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 07 08:58:37.416050 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.416044    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 07 08:58:37.416156 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.416052    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 07 08:58:37.416912 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.416903    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 07 08:58:37.416912 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.416912    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 07 08:58:37.420550 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.420539    2574 server.go:1268] "Started kubelet"
Sep 07 08:58:37.420656 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.420621    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 07 08:58:37.420729 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.420678    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 07 08:58:37.420729 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.420633    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 07 08:58:37.421294 ip-10-0-141-81 systemd[1]: Started Kubernetes Kubelet.
Sep 07 08:58:37.424999 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.424976    2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 07 08:58:37.426463 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.426436    2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 07 08:58:37.427015 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.426995    2574 server.go:310] "Adding debug handlers to kubelet server"
Sep 07 08:58:37.428944 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.428926    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-81.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 07 08:58:37.429688 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.429664    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 07 08:58:37.429688 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.429665    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-81.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 07 08:58:37.429975 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.428894    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-81.ec2.internal.18d2fe37c02fa962  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-81.ec2.internal,UID:ip-10-0-141-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-81.ec2.internal,},FirstTimestamp:2026-09-07 08:58:37.420513634 +0000 UTC m=+0.384429846,LastTimestamp:2026-09-07 08:58:37.420513634 +0000 UTC m=+0.384429846,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-81.ec2.internal,}"
Sep 07 08:58:37.433016 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.432998    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 07 08:58:37.433837 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.433821    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 07 08:58:37.433934 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.433908    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:37.434542 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.434524    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 07 08:58:37.434542 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.434524    2574 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 07 08:58:37.434679 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.434550    2574 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 07 08:58:37.434739 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.434721    2574 reconstruct.go:97] "Volume reconstruction finished"
Sep 07 08:58:37.434831 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.434781    2574 reconciler.go:29] "Reconciler: start to sync state"
Sep 07 08:58:37.434999 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.434985    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 07 08:58:37.435061 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.435000    2574 factory.go:55] Registering systemd factory
Sep 07 08:58:37.435061 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.435009    2574 factory.go:223] Registration of the systemd container factory successfully
Sep 07 08:58:37.435233 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.435219    2574 factory.go:153] Registering CRI-O factory
Sep 07 08:58:37.435303 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.435235    2574 factory.go:223] Registration of the crio container factory successfully
Sep 07 08:58:37.435303 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.435252    2574 factory.go:103] Registering Raw factory
Sep 07 08:58:37.435303 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.435264    2574 manager.go:1196] Started watching for new ooms in manager
Sep 07 08:58:37.436230 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.436216    2574 manager.go:349] Starting recovery of all containers
Sep 07 08:58:37.442904 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.442883    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b2vbf"
Sep 07 08:58:37.443602 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.443574    2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-81.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 07 08:58:37.443602 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.443579    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 07 08:58:37.446437 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.446421    2574 manager.go:354] Recovery completed
Sep 07 08:58:37.450562 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.450545    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b2vbf"
Sep 07 08:58:37.450772 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.450760    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 08:58:37.452299 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.452282    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 08:58:37.452356 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.452317    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 08:58:37.452356 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.452333    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientPID"
Sep 07 08:58:37.452929 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.452915    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 07 08:58:37.452929 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.452928    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 07 08:58:37.453050 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.452939    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 08:58:37.453050 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.452942    2574 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 08:58:37.453619 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.453606    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 08:58:37.453656 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.453630    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 08:58:37.453656 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.453640    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientPID"
Sep 07 08:58:37.454894 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.454882    2574 policy_none.go:49] "None policy: Start"
Sep 07 08:58:37.454945 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.454898    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 07 08:58:37.454945 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.454908    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 07 08:58:37.456780 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.456770    2574 policy_none.go:47] "Start"
Sep 07 08:58:37.503394 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.503275    2574 manager.go:337] "Starting Device Plugin manager"
Sep 07 08:58:37.519809 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.503421    2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 07 08:58:37.519809 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.503434    2574 server.go:87] "Starting device plugin registration server"
Sep 07 08:58:37.519809 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.503758    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 07 08:58:37.519809 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.503770    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 07 08:58:37.519809 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.503832    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 07 08:58:37.519809 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.503920    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 07 08:58:37.519809 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.503929    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 07 08:58:37.519809 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.504458    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 07 08:58:37.519809 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.504511    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:37.534110 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.534084    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 07 08:58:37.535333 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.535313    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 07 08:58:37.535407 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.535355    2574 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 07 08:58:37.535407 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.535375    2574 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 07 08:58:37.535503 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.535432    2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 07 08:58:37.537898 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.537875    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 08:58:37.604798 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.604745    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 08:58:37.605512 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.605499    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 08:58:37.605560 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.605525    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 08:58:37.605560 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.605536    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientPID"
Sep 07 08:58:37.605621 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.605562    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.616129 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.616114    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.616174 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.616133    2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-81.ec2.internal\": node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:37.635762 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.635740    2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal"]
Sep 07 08:58:37.635851 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.635806    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 08:58:37.636963 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.636937    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 08:58:37.637055 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.636966    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 08:58:37.637055 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.636981    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientPID"
Sep 07 08:58:37.637055 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.637053    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:37.639105 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.639091    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 08:58:37.639221 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.639206    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.639268 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.639237    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 08:58:37.639799 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.639782    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 08:58:37.639879 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.639785    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 08:58:37.639879 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.639817    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 08:58:37.639879 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.639824    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 08:58:37.639879 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.639832    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientPID"
Sep 07 08:58:37.639879 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.639838    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientPID"
Sep 07 08:58:37.641921 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.641905    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.641996 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.641933    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 08:58:37.643567 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.643550    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 08:58:37.643660 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.643579    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 08:58:37.643660 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.643591    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeHasSufficientPID"
Sep 07 08:58:37.666272 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.666255    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-81.ec2.internal\" not found" node="ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.670404 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.670387    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-81.ec2.internal\" not found" node="ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.736126 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.736109    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9132b1637729b3158d4294075bfe2281-config\") pod \"kube-apiserver-proxy-ip-10-0-141-81.ec2.internal\" (UID: \"9132b1637729b3158d4294075bfe2281\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.736223 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.736131    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/501b893153a8107dd528995fa664b1c7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal\" (UID: \"501b893153a8107dd528995fa664b1c7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.736223 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.736148    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/501b893153a8107dd528995fa664b1c7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal\" (UID: \"501b893153a8107dd528995fa664b1c7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.737161 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.737147    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:37.836538 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.836508    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/501b893153a8107dd528995fa664b1c7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal\" (UID: \"501b893153a8107dd528995fa664b1c7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.836623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.836546    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/501b893153a8107dd528995fa664b1c7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal\" (UID: \"501b893153a8107dd528995fa664b1c7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.836623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.836570    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9132b1637729b3158d4294075bfe2281-config\") pod \"kube-apiserver-proxy-ip-10-0-141-81.ec2.internal\" (UID: \"9132b1637729b3158d4294075bfe2281\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.836623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.836617    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/501b893153a8107dd528995fa664b1c7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal\" (UID: \"501b893153a8107dd528995fa664b1c7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.836759 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.836621    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9132b1637729b3158d4294075bfe2281-config\") pod \"kube-apiserver-proxy-ip-10-0-141-81.ec2.internal\" (UID: \"9132b1637729b3158d4294075bfe2281\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.836759 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.836629    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/501b893153a8107dd528995fa664b1c7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal\" (UID: \"501b893153a8107dd528995fa664b1c7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.837566 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.837550    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:37.938234 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:37.938217    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:37.969392 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.969367    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:37.973106 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:37.973090    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:38.038964 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:38.038936    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:38.139383 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:38.139360    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:38.239797 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:38.239746    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:38.340203 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:38.340178    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:38.366329 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:38.366309    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 07 08:58:38.366787 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:38.366435    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 07 08:58:38.440564 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:38.440543    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:38.454033 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:38.454008    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-06 08:53:38 +0000 UTC" deadline="2028-02-04 09:00:02.746227933 +0000 UTC"
Sep 07 08:58:38.454033 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:38.454031    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12360h1m24.292200094s"
Sep 07 08:58:38.541369 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:38.541322    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:38.561307 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:38.561293    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 08:58:38.642087 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:38.642067    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:38.742624 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:38.742591    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:38.783776 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:38.783704    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod501b893153a8107dd528995fa664b1c7.slice/crio-14a5ff366f5d724f9c1b0e0ecc1d92e5c350638760ac4da78b24f0e4c70aef5f WatchSource:0}: Error finding container 14a5ff366f5d724f9c1b0e0ecc1d92e5c350638760ac4da78b24f0e4c70aef5f: Status 404 returned error can't find the container with id 14a5ff366f5d724f9c1b0e0ecc1d92e5c350638760ac4da78b24f0e4c70aef5f
Sep 07 08:58:38.784166 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:38.784136    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9132b1637729b3158d4294075bfe2281.slice/crio-4a8c74debdc99220ca4cff882513386a2b090c68fc2becdcc139b6e68f97e859 WatchSource:0}: Error finding container 4a8c74debdc99220ca4cff882513386a2b090c68fc2becdcc139b6e68f97e859: Status 404 returned error can't find the container with id 4a8c74debdc99220ca4cff882513386a2b090c68fc2becdcc139b6e68f97e859
Sep 07 08:58:38.789644 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:38.789626    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 08:58:38.842837 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:38.842808    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-81.ec2.internal\" not found"
Sep 07 08:58:38.854736 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:38.854718    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 08:58:38.921467 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:38.921446    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 08:58:38.934956 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:38.934931    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:38.947773 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:38.947755    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 08:58:38.948870 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:38.948856    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal"
Sep 07 08:58:38.955283 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:38.955270    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 08:58:39.355291 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.355260    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 08:58:39.411376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.411348    2574 apiserver.go:52] "Watching apiserver"
Sep 07 08:58:39.423616 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.423588    2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 07 08:58:39.425823 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.425792    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-zdlqw","kube-system/global-pull-secret-syncer-z54bd","kube-system/konnectivity-agent-czgxz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6","openshift-cluster-node-tuning-operator/tuned-m24pj","openshift-dns/node-resolver-rpdbr","openshift-multus/multus-additional-cni-plugins-tjrhl","openshift-network-operator/iptables-alerter-tfszw","kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal","openshift-image-registry/node-ca-4s9pv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal","openshift-multus/multus-l8xfv","openshift-multus/network-metrics-daemon-c594r","openshift-network-diagnostics/network-check-target-sgxq8"]
Sep 07 08:58:39.428589 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.428565    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.430993 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.430973    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:39.431095 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:39.431042    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:58:39.431173 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.431114    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 07 08:58:39.431316 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.431297    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 07 08:58:39.431441 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.431380    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 07 08:58:39.431496 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.431475    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 07 08:58:39.431583 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.431563    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-r2ppj\""
Sep 07 08:58:39.431729 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.431623    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 07 08:58:39.433666 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.433641    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-czgxz"
Sep 07 08:58:39.433987 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.433960    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 07 08:58:39.435807 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.435768    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 07 08:58:39.435905 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.435851    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-w4g7r\""
Sep 07 08:58:39.436030 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.436014    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.436100 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.436061    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 07 08:58:39.438409 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.438261    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 07 08:58:39.438409 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.438321    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mxdzv\""
Sep 07 08:58:39.438558 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.438548    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 07 08:58:39.438778 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.438264    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 07 08:58:39.438928 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.438910    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.440765 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.440745    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 07 08:58:39.441100 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.441078    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 07 08:58:39.441541 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.441524    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kssgh\""
Sep 07 08:58:39.442090 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.441869    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rpdbr"
Sep 07 08:58:39.442865 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.442845    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 08:58:39.443458 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.443440    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 07 08:58:39.443697 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.443682    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hzjn7\""
Sep 07 08:58:39.443914 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.443893    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 07 08:58:39.444171 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.444152    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:39.444284 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:39.444257    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:58:39.445678 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.445657    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/78b45f22-f23b-46a7-b3f6-21a960bf365e-konnectivity-ca\") pod \"konnectivity-agent-czgxz\" (UID: \"78b45f22-f23b-46a7-b3f6-21a960bf365e\") " pod="kube-system/konnectivity-agent-czgxz"
Sep 07 08:58:39.445783 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.445691    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-sys-fs\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.445783 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.445721    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrkvz\" (UniqueName: \"kubernetes.io/projected/99a03c01-5968-408a-a4ca-957acdce1bd7-kube-api-access-rrkvz\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.445783 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.445753    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-socket-dir\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.445783 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.445777    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-device-dir\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.445974 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.445862    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.445974 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.445904    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl27q\" (UniqueName: \"kubernetes.io/projected/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-kube-api-access-zl27q\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.445974 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.445953    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99a03c01-5968-408a-a4ca-957acdce1bd7-system-cni-dir\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.446106 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.446004    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99a03c01-5968-408a-a4ca-957acdce1bd7-cnibin\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.446106 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.446042    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99a03c01-5968-408a-a4ca-957acdce1bd7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.446106 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.446086    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs\") pod \"network-metrics-daemon-c594r\" (UID: \"2aee0981-afb8-47c7-8484-142812fc6d42\") " pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:39.446336 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.446113    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-registration-dir\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.446336 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.446155    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.446336 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.446199    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99a03c01-5968-408a-a4ca-957acdce1bd7-cni-binary-copy\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.446336 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.446238    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/99a03c01-5968-408a-a4ca-957acdce1bd7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.446336 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.446266    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/99a03c01-5968-408a-a4ca-957acdce1bd7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.446336 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.446303    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqfgb\" (UniqueName: \"kubernetes.io/projected/2aee0981-afb8-47c7-8484-142812fc6d42-kube-api-access-zqfgb\") pod \"network-metrics-daemon-c594r\" (UID: \"2aee0981-afb8-47c7-8484-142812fc6d42\") " pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:39.446633 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.446338    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/78b45f22-f23b-46a7-b3f6-21a960bf365e-agent-certs\") pod \"konnectivity-agent-czgxz\" (UID: \"78b45f22-f23b-46a7-b3f6-21a960bf365e\") " pod="kube-system/konnectivity-agent-czgxz"
Sep 07 08:58:39.446633 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.446586    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.448542 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.448523    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 07 08:58:39.448629 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.448527    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 07 08:58:39.448715 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.448697    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r9b7x\""
Sep 07 08:58:39.451930 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.450853    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:39.451930 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:39.450959    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:58:39.451930 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.451204    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 07 08:58:39.451930 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.451447    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 07 08:58:39.451930 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.451483    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 07 08:58:39.451930 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.451885    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 07 08:58:39.454333 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.454220    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4s9pv"
Sep 07 08:58:39.456545 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.456468    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 07 08:58:39.456545 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.456487    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 07 08:58:39.456689 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.456627    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 07 08:58:39.456857 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.456837    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fbrg4\""
Sep 07 08:58:39.457663 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.457228    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tfszw"
Sep 07 08:58:39.459253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.459212    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9m8p9\""
Sep 07 08:58:39.459507 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.459493    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 07 08:58:39.459664 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.459647    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.459749 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.459733    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 07 08:58:39.459889 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.459650    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 07 08:58:39.461734 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.461713    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 07 08:58:39.461985 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.461965    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5plsn\""
Sep 07 08:58:39.471889 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.471864    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7zxkc"
Sep 07 08:58:39.479423 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.479291    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7zxkc"
Sep 07 08:58:39.536396 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.536375    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 07 08:58:39.541638 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.541589    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal" event={"ID":"501b893153a8107dd528995fa664b1c7","Type":"ContainerStarted","Data":"14a5ff366f5d724f9c1b0e0ecc1d92e5c350638760ac4da78b24f0e4c70aef5f"}
Sep 07 08:58:39.543418 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.543393    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal" event={"ID":"9132b1637729b3158d4294075bfe2281","Type":"ContainerStarted","Data":"4a8c74debdc99220ca4cff882513386a2b090c68fc2becdcc139b6e68f97e859"}
Sep 07 08:58:39.547025 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.546712    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99a03c01-5968-408a-a4ca-957acdce1bd7-cni-binary-copy\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.547025 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.546767    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/78b45f22-f23b-46a7-b3f6-21a960bf365e-agent-certs\") pod \"konnectivity-agent-czgxz\" (UID: \"78b45f22-f23b-46a7-b3f6-21a960bf365e\") " pod="kube-system/konnectivity-agent-czgxz"
Sep 07 08:58:39.547025 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.546795    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-sys-fs\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.547025 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.546823    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-sysconfig\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.547025 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.546847    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-run-ovn\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.547025 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.546871    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-node-log\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.547025 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.546895    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eebe8c71-1cd7-4040-af2d-7b636fd884c2-ovn-node-metrics-cert\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.547025 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.546918    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-sys\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.547025 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.546940    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-host\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.547025 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.546965    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:39.547025 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.546988    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-etc-kubernetes\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.547025 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547016    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547041    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl27q\" (UniqueName: \"kubernetes.io/projected/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-kube-api-access-zl27q\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547066    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-run-openvswitch\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547091    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zcd2\" (UniqueName: \"kubernetes.io/projected/eebe8c71-1cd7-4040-af2d-7b636fd884c2-kube-api-access-6zcd2\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547114    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsqpz\" (UniqueName: \"kubernetes.io/projected/23a08429-ee73-45ad-8a29-bbcf1a359f02-kube-api-access-nsqpz\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547162    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99a03c01-5968-408a-a4ca-957acdce1bd7-system-cni-dir\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547196    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs\") pod \"network-metrics-daemon-c594r\" (UID: \"2aee0981-afb8-47c7-8484-142812fc6d42\") " pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547220    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-registration-dir\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547255    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eebe8c71-1cd7-4040-af2d-7b636fd884c2-ovnkube-script-lib\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547278    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fa333c43-6b4b-4023-8665-bd82f6943f20-serviceca\") pod \"node-ca-4s9pv\" (UID: \"fa333c43-6b4b-4023-8665-bd82f6943f20\") " pod="openshift-image-registry/node-ca-4s9pv"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547304    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/99a03c01-5968-408a-a4ca-957acdce1bd7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547330    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/78b45f22-f23b-46a7-b3f6-21a960bf365e-konnectivity-ca\") pod \"konnectivity-agent-czgxz\" (UID: \"78b45f22-f23b-46a7-b3f6-21a960bf365e\") " pod="kube-system/konnectivity-agent-czgxz"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547354    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-run\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547379    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx6mf\" (UniqueName: \"kubernetes.io/projected/5c88d609-ca70-4755-9e0d-b28846ba8ca0-kube-api-access-hx6mf\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547403    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-var-lib-cni-bin\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/23a08429-ee73-45ad-8a29-bbcf1a359f02-multus-daemon-config\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.547712 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547472    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-socket-dir\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547497    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-run-netns\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547521    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqsmx\" (UniqueName: \"kubernetes.io/projected/fa333c43-6b4b-4023-8665-bd82f6943f20-kube-api-access-fqsmx\") pod \"node-ca-4s9pv\" (UID: \"fa333c43-6b4b-4023-8665-bd82f6943f20\") " pod="openshift-image-registry/node-ca-4s9pv"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547549    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-run-k8s-cni-cncf-io\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547582    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-hostroot\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547607    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e4df05be-d0fe-470e-bd44-7f506c6697b1-iptables-alerter-script\") pod \"iptables-alerter-tfszw\" (UID: \"e4df05be-d0fe-470e-bd44-7f506c6697b1\") " pod="openshift-network-operator/iptables-alerter-tfszw"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547641    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/40ddcfef-57f6-46fd-8d22-d4d0b5da9647-hosts-file\") pod \"node-resolver-rpdbr\" (UID: \"40ddcfef-57f6-46fd-8d22-d4d0b5da9647\") " pod="openshift-dns/node-resolver-rpdbr"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547672    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-var-lib-openvswitch\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547698    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547726    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqfgb\" (UniqueName: \"kubernetes.io/projected/2aee0981-afb8-47c7-8484-142812fc6d42-kube-api-access-zqfgb\") pod \"network-metrics-daemon-c594r\" (UID: \"2aee0981-afb8-47c7-8484-142812fc6d42\") " pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547752    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4df05be-d0fe-470e-bd44-7f506c6697b1-host-slash\") pod \"iptables-alerter-tfszw\" (UID: \"e4df05be-d0fe-470e-bd44-7f506c6697b1\") " pod="openshift-network-operator/iptables-alerter-tfszw"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547777    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7rcv\" (UniqueName: \"kubernetes.io/projected/e4df05be-d0fe-470e-bd44-7f506c6697b1-kube-api-access-d7rcv\") pod \"iptables-alerter-tfszw\" (UID: \"e4df05be-d0fe-470e-bd44-7f506c6697b1\") " pod="openshift-network-operator/iptables-alerter-tfszw"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547820    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-cni-bin\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547846    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-kubelet-config\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547875    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-var-lib-kubelet\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547885    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99a03c01-5968-408a-a4ca-957acdce1bd7-cni-binary-copy\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.548532 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547905    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrkvz\" (UniqueName: \"kubernetes.io/projected/99a03c01-5968-408a-a4ca-957acdce1bd7-kube-api-access-rrkvz\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547931    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-device-dir\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547955    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-sysctl-d\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.547994    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40ddcfef-57f6-46fd-8d22-d4d0b5da9647-tmp-dir\") pod \"node-resolver-rpdbr\" (UID: \"40ddcfef-57f6-46fd-8d22-d4d0b5da9647\") " pod="openshift-dns/node-resolver-rpdbr"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548019    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-etc-openvswitch\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548044    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-multus-cni-dir\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548069    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-sysctl-conf\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548093    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-tuned\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548119    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-systemd-units\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548151    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eebe8c71-1cd7-4040-af2d-7b636fd884c2-ovnkube-config\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548177    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-run-multus-certs\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548207    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-lib-modules\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548232    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-var-lib-kubelet\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548257    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-system-cni-dir\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548282    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-cnibin\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548310    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-var-lib-cni-multus\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548337    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/99a03c01-5968-408a-a4ca-957acdce1bd7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.549376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548360    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c88d609-ca70-4755-9e0d-b28846ba8ca0-tmp\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548394    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548421    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548450    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-kubernetes\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548475    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-slash\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548500    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-run-systemd\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548524    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23a08429-ee73-45ad-8a29-bbcf1a359f02-cni-binary-copy\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548548    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-modprobe-d\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548577    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hmx8\" (UniqueName: \"kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8\") pod \"network-check-target-sgxq8\" (UID: \"e8f68cb8-c496-4c1e-b79d-081e5fc8d121\") " pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548604    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfwfp\" (UniqueName: \"kubernetes.io/projected/40ddcfef-57f6-46fd-8d22-d4d0b5da9647-kube-api-access-lfwfp\") pod \"node-resolver-rpdbr\" (UID: \"40ddcfef-57f6-46fd-8d22-d4d0b5da9647\") " pod="openshift-dns/node-resolver-rpdbr"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-kubelet\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548657    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-log-socket\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548682    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eebe8c71-1cd7-4040-af2d-7b636fd884c2-env-overrides\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548706    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa333c43-6b4b-4023-8665-bd82f6943f20-host\") pod \"node-ca-4s9pv\" (UID: \"fa333c43-6b4b-4023-8665-bd82f6943f20\") " pod="openshift-image-registry/node-ca-4s9pv"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548731    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-multus-socket-dir-parent\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.548760    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99a03c01-5968-408a-a4ca-957acdce1bd7-cnibin\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.550178 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.549749    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99a03c01-5968-408a-a4ca-957acdce1bd7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.550971 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.549825    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-systemd\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.550971 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.549866    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-cni-netd\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.550971 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.549905    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-dbus\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:39.550971 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.549941    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-run-netns\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.550971 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.549969    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-multus-conf-dir\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.550971 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.550231    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99a03c01-5968-408a-a4ca-957acdce1bd7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.550971 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.550776    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-device-dir\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.551974 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.551497    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/99a03c01-5968-408a-a4ca-957acdce1bd7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.551974 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.551647    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 07 08:58:39.551974 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.551817    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-sys-fs\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.552216 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.551990    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.552400 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.552378    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99a03c01-5968-408a-a4ca-957acdce1bd7-system-cni-dir\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.552950 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.552927    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/99a03c01-5968-408a-a4ca-957acdce1bd7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.553470 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.553448    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/78b45f22-f23b-46a7-b3f6-21a960bf365e-konnectivity-ca\") pod \"konnectivity-agent-czgxz\" (UID: \"78b45f22-f23b-46a7-b3f6-21a960bf365e\") " pod="kube-system/konnectivity-agent-czgxz"
Sep 07 08:58:39.553673 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.553653    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-socket-dir\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.553774 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.553723    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99a03c01-5968-408a-a4ca-957acdce1bd7-cnibin\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.553834 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.553815    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-registration-dir\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.553997 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.553974    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.554563 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:39.554540    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:58:39.554664 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:39.554651    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs podName:2aee0981-afb8-47c7-8484-142812fc6d42 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:40.054605075 +0000 UTC m=+3.018521281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs") pod "network-metrics-daemon-c594r" (UID: "2aee0981-afb8-47c7-8484-142812fc6d42") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:58:39.563352 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.563329    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/78b45f22-f23b-46a7-b3f6-21a960bf365e-agent-certs\") pod \"konnectivity-agent-czgxz\" (UID: \"78b45f22-f23b-46a7-b3f6-21a960bf365e\") " pod="kube-system/konnectivity-agent-czgxz"
Sep 07 08:58:39.564096 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.564068    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl27q\" (UniqueName: \"kubernetes.io/projected/90f4682b-cad1-4e6c-bda1-6e04cb6839cd-kube-api-access-zl27q\") pod \"aws-ebs-csi-driver-node-7ckm6\" (UID: \"90f4682b-cad1-4e6c-bda1-6e04cb6839cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.565763 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.565737    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrkvz\" (UniqueName: \"kubernetes.io/projected/99a03c01-5968-408a-a4ca-957acdce1bd7-kube-api-access-rrkvz\") pod \"multus-additional-cni-plugins-tjrhl\" (UID: \"99a03c01-5968-408a-a4ca-957acdce1bd7\") " pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.566644 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.566622    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqfgb\" (UniqueName: \"kubernetes.io/projected/2aee0981-afb8-47c7-8484-142812fc6d42-kube-api-access-zqfgb\") pod \"network-metrics-daemon-c594r\" (UID: \"2aee0981-afb8-47c7-8484-142812fc6d42\") " pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:39.650926 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.650842    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-dbus\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:39.650926 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.650885    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-run-netns\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.650926 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.650910    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-multus-conf-dir\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.650926 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.650929    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-sysconfig\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.650959    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-run-ovn\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.650974    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-node-log\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.650988    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eebe8c71-1cd7-4040-af2d-7b636fd884c2-ovn-node-metrics-cert\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651010    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-sys\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651031    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-host\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651056    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651080    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-etc-kubernetes\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651100    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-run-openvswitch\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651116    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zcd2\" (UniqueName: \"kubernetes.io/projected/eebe8c71-1cd7-4040-af2d-7b636fd884c2-kube-api-access-6zcd2\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651131    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsqpz\" (UniqueName: \"kubernetes.io/projected/23a08429-ee73-45ad-8a29-bbcf1a359f02-kube-api-access-nsqpz\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651158    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eebe8c71-1cd7-4040-af2d-7b636fd884c2-ovnkube-script-lib\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651172    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fa333c43-6b4b-4023-8665-bd82f6943f20-serviceca\") pod \"node-ca-4s9pv\" (UID: \"fa333c43-6b4b-4023-8665-bd82f6943f20\") " pod="openshift-image-registry/node-ca-4s9pv"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651187    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-run\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651211    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx6mf\" (UniqueName: \"kubernetes.io/projected/5c88d609-ca70-4755-9e0d-b28846ba8ca0-kube-api-access-hx6mf\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651229    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-var-lib-cni-bin\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.651253 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651251    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/23a08429-ee73-45ad-8a29-bbcf1a359f02-multus-daemon-config\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651277    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-run-netns\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651301    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqsmx\" (UniqueName: \"kubernetes.io/projected/fa333c43-6b4b-4023-8665-bd82f6943f20-kube-api-access-fqsmx\") pod \"node-ca-4s9pv\" (UID: \"fa333c43-6b4b-4023-8665-bd82f6943f20\") " pod="openshift-image-registry/node-ca-4s9pv"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651317    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-run-k8s-cni-cncf-io\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651340    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-hostroot\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651359    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e4df05be-d0fe-470e-bd44-7f506c6697b1-iptables-alerter-script\") pod \"iptables-alerter-tfszw\" (UID: \"e4df05be-d0fe-470e-bd44-7f506c6697b1\") " pod="openshift-network-operator/iptables-alerter-tfszw"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651374    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/40ddcfef-57f6-46fd-8d22-d4d0b5da9647-hosts-file\") pod \"node-resolver-rpdbr\" (UID: \"40ddcfef-57f6-46fd-8d22-d4d0b5da9647\") " pod="openshift-dns/node-resolver-rpdbr"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651391    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-var-lib-openvswitch\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651408    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4df05be-d0fe-470e-bd44-7f506c6697b1-host-slash\") pod \"iptables-alerter-tfszw\" (UID: \"e4df05be-d0fe-470e-bd44-7f506c6697b1\") " pod="openshift-network-operator/iptables-alerter-tfszw"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651423    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7rcv\" (UniqueName: \"kubernetes.io/projected/e4df05be-d0fe-470e-bd44-7f506c6697b1-kube-api-access-d7rcv\") pod \"iptables-alerter-tfszw\" (UID: \"e4df05be-d0fe-470e-bd44-7f506c6697b1\") " pod="openshift-network-operator/iptables-alerter-tfszw"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651446    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-cni-bin\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651466    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-kubelet-config\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651482    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-var-lib-kubelet\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651499    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-sysctl-d\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651526    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40ddcfef-57f6-46fd-8d22-d4d0b5da9647-tmp-dir\") pod \"node-resolver-rpdbr\" (UID: \"40ddcfef-57f6-46fd-8d22-d4d0b5da9647\") " pod="openshift-dns/node-resolver-rpdbr"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651553    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-etc-openvswitch\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651569    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-multus-cni-dir\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.652076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651584    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-sysctl-conf\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651599    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-tuned\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651630    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-systemd-units\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651658    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eebe8c71-1cd7-4040-af2d-7b636fd884c2-ovnkube-config\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651673    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-run-multus-certs\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651688    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-lib-modules\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651713    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-var-lib-kubelet\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651740    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-system-cni-dir\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651763    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-cnibin\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651779    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-var-lib-cni-multus\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651815    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c88d609-ca70-4755-9e0d-b28846ba8ca0-tmp\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651854    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651881    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651909    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-kubernetes\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651936    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-slash\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651962    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-run-systemd\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.651987    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23a08429-ee73-45ad-8a29-bbcf1a359f02-cni-binary-copy\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.652909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652013    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-modprobe-d\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652054    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hmx8\" (UniqueName: \"kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8\") pod \"network-check-target-sgxq8\" (UID: \"e8f68cb8-c496-4c1e-b79d-081e5fc8d121\") " pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652081    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfwfp\" (UniqueName: \"kubernetes.io/projected/40ddcfef-57f6-46fd-8d22-d4d0b5da9647-kube-api-access-lfwfp\") pod \"node-resolver-rpdbr\" (UID: \"40ddcfef-57f6-46fd-8d22-d4d0b5da9647\") " pod="openshift-dns/node-resolver-rpdbr"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652108    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-kubelet\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652134    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-log-socket\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652172    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eebe8c71-1cd7-4040-af2d-7b636fd884c2-env-overrides\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652198    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa333c43-6b4b-4023-8665-bd82f6943f20-host\") pod \"node-ca-4s9pv\" (UID: \"fa333c43-6b4b-4023-8665-bd82f6943f20\") " pod="openshift-image-registry/node-ca-4s9pv"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652224    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-multus-socket-dir-parent\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652255    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-systemd\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652282    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-cni-netd\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652392    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-cni-netd\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652460    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-dbus\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652502    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-run-netns\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652539    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-multus-conf-dir\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652585    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-sysconfig\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652622    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-run-ovn\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.652679    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-node-log\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.653021    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-etc-openvswitch\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.653730 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.653092    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-sys\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.654666 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.653137    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-host\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.654666 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:39.653214    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 08:58:39.654666 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:39.653262    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret podName:e3bc99a4-33c4-4a42-a55c-ef8de46f9502 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:40.153242881 +0000 UTC m=+3.117159086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret") pod "global-pull-secret-syncer-z54bd" (UID: "e3bc99a4-33c4-4a42-a55c-ef8de46f9502") : object "kube-system"/"original-pull-secret" not registered
Sep 07 08:58:39.654666 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.653418    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-etc-kubernetes\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.654666 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.653463    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-run-openvswitch\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.654666 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.654252    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/eebe8c71-1cd7-4040-af2d-7b636fd884c2-ovnkube-script-lib\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.654666 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.654642    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fa333c43-6b4b-4023-8665-bd82f6943f20-serviceca\") pod \"node-ca-4s9pv\" (UID: \"fa333c43-6b4b-4023-8665-bd82f6943f20\") " pod="openshift-image-registry/node-ca-4s9pv"
Sep 07 08:58:39.655040 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.654710    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-run\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.655040 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.654964    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-var-lib-cni-bin\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655227    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655273    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4df05be-d0fe-470e-bd44-7f506c6697b1-host-slash\") pod \"iptables-alerter-tfszw\" (UID: \"e4df05be-d0fe-470e-bd44-7f506c6697b1\") " pod="openshift-network-operator/iptables-alerter-tfszw"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655310    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-lib-modules\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655332    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-run-netns\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655316    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-var-lib-kubelet\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655407    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-multus-cni-dir\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655413    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-system-cni-dir\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655468    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-cnibin\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655475    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/23a08429-ee73-45ad-8a29-bbcf1a359f02-multus-daemon-config\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655512    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-var-lib-cni-multus\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655554    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-sysctl-conf\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655575    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/eebe8c71-1cd7-4040-af2d-7b636fd884c2-ovn-node-metrics-cert\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655581    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-run-k8s-cni-cncf-io\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655634    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-hostroot\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655743    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-cni-bin\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655840    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-kubelet-config\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655880    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/40ddcfef-57f6-46fd-8d22-d4d0b5da9647-hosts-file\") pod \"node-resolver-rpdbr\" (UID: \"40ddcfef-57f6-46fd-8d22-d4d0b5da9647\") " pod="openshift-dns/node-resolver-rpdbr"
Sep 07 08:58:39.656613 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.655893    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-var-lib-kubelet\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656014    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-sysctl-d\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656044    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-slash\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656099    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-run-systemd\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656213    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e4df05be-d0fe-470e-bd44-7f506c6697b1-iptables-alerter-script\") pod \"iptables-alerter-tfszw\" (UID: \"e4df05be-d0fe-470e-bd44-7f506c6697b1\") " pod="openshift-network-operator/iptables-alerter-tfszw"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656274    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-systemd-units\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656321    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-var-lib-openvswitch\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656331    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/eebe8c71-1cd7-4040-af2d-7b636fd884c2-env-overrides\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656367    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40ddcfef-57f6-46fd-8d22-d4d0b5da9647-tmp-dir\") pod \"node-resolver-rpdbr\" (UID: \"40ddcfef-57f6-46fd-8d22-d4d0b5da9647\") " pod="openshift-dns/node-resolver-rpdbr"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656375    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/eebe8c71-1cd7-4040-af2d-7b636fd884c2-ovnkube-config\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656392    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-kubelet\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656418    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-log-socket\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656373    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-host-run-multus-certs\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656468    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-systemd\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656472    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23a08429-ee73-45ad-8a29-bbcf1a359f02-multus-socket-dir-parent\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656526    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa333c43-6b4b-4023-8665-bd82f6943f20-host\") pod \"node-ca-4s9pv\" (UID: \"fa333c43-6b4b-4023-8665-bd82f6943f20\") " pod="openshift-image-registry/node-ca-4s9pv"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656532    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eebe8c71-1cd7-4040-af2d-7b636fd884c2-host-run-ovn-kubernetes\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656572    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23a08429-ee73-45ad-8a29-bbcf1a359f02-cni-binary-copy\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.657593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656577    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-kubernetes\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.658564 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.656622    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-modprobe-d\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.658564 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.658408    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c88d609-ca70-4755-9e0d-b28846ba8ca0-tmp\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.659330 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.659311    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5c88d609-ca70-4755-9e0d-b28846ba8ca0-etc-tuned\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.661727 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:39.661692    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 08:58:39.661850 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:39.661731    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 08:58:39.661850 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:39.661742    2574 projected.go:196] Error preparing data for projected volume kube-api-access-6hmx8 for pod openshift-network-diagnostics/network-check-target-sgxq8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:58:39.661850 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:39.661795    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8 podName:e8f68cb8-c496-4c1e-b79d-081e5fc8d121 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:40.161782896 +0000 UTC m=+3.125699096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6hmx8" (UniqueName: "kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8") pod "network-check-target-sgxq8" (UID: "e8f68cb8-c496-4c1e-b79d-081e5fc8d121") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:58:39.665455 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.665430    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7rcv\" (UniqueName: \"kubernetes.io/projected/e4df05be-d0fe-470e-bd44-7f506c6697b1-kube-api-access-d7rcv\") pod \"iptables-alerter-tfszw\" (UID: \"e4df05be-d0fe-470e-bd44-7f506c6697b1\") " pod="openshift-network-operator/iptables-alerter-tfszw"
Sep 07 08:58:39.666095 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.666065    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfwfp\" (UniqueName: \"kubernetes.io/projected/40ddcfef-57f6-46fd-8d22-d4d0b5da9647-kube-api-access-lfwfp\") pod \"node-resolver-rpdbr\" (UID: \"40ddcfef-57f6-46fd-8d22-d4d0b5da9647\") " pod="openshift-dns/node-resolver-rpdbr"
Sep 07 08:58:39.666226 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.666197    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsqpz\" (UniqueName: \"kubernetes.io/projected/23a08429-ee73-45ad-8a29-bbcf1a359f02-kube-api-access-nsqpz\") pod \"multus-l8xfv\" (UID: \"23a08429-ee73-45ad-8a29-bbcf1a359f02\") " pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:39.667065 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.667042    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqsmx\" (UniqueName: \"kubernetes.io/projected/fa333c43-6b4b-4023-8665-bd82f6943f20-kube-api-access-fqsmx\") pod \"node-ca-4s9pv\" (UID: \"fa333c43-6b4b-4023-8665-bd82f6943f20\") " pod="openshift-image-registry/node-ca-4s9pv"
Sep 07 08:58:39.667484 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.667464    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zcd2\" (UniqueName: \"kubernetes.io/projected/eebe8c71-1cd7-4040-af2d-7b636fd884c2-kube-api-access-6zcd2\") pod \"ovnkube-node-zdlqw\" (UID: \"eebe8c71-1cd7-4040-af2d-7b636fd884c2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.667978 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.667957    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx6mf\" (UniqueName: \"kubernetes.io/projected/5c88d609-ca70-4755-9e0d-b28846ba8ca0-kube-api-access-hx6mf\") pod \"tuned-m24pj\" (UID: \"5c88d609-ca70-4755-9e0d-b28846ba8ca0\") " pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.741165 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.741133    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tjrhl"
Sep 07 08:58:39.752119 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.752095    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-czgxz"
Sep 07 08:58:39.760263 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.760242    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6"
Sep 07 08:58:39.769205 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.769184    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m24pj"
Sep 07 08:58:39.775808 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.775789    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rpdbr"
Sep 07 08:58:39.784708 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.784687    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:58:39.792786 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.792768    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4s9pv"
Sep 07 08:58:39.801534 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.801516    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tfszw"
Sep 07 08:58:39.808419 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:39.808390    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l8xfv"
Sep 07 08:58:40.056962 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:40.056923    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs\") pod \"network-metrics-daemon-c594r\" (UID: \"2aee0981-afb8-47c7-8484-142812fc6d42\") " pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:40.057153 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:40.057106    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:58:40.057216 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:40.057191    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs podName:2aee0981-afb8-47c7-8484-142812fc6d42 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:41.057168906 +0000 UTC m=+4.021085119 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs") pod "network-metrics-daemon-c594r" (UID: "2aee0981-afb8-47c7-8484-142812fc6d42") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:58:40.157561 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:40.157528    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:40.157763 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:40.157697    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 08:58:40.157836 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:40.157800    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret podName:e3bc99a4-33c4-4a42-a55c-ef8de46f9502 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:41.157778356 +0000 UTC m=+4.121694569 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret") pod "global-pull-secret-syncer-z54bd" (UID: "e3bc99a4-33c4-4a42-a55c-ef8de46f9502") : object "kube-system"/"original-pull-secret" not registered
Sep 07 08:58:40.258343 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:40.258296    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hmx8\" (UniqueName: \"kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8\") pod \"network-check-target-sgxq8\" (UID: \"e8f68cb8-c496-4c1e-b79d-081e5fc8d121\") " pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:40.258548 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:40.258520    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 08:58:40.258618 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:40.258560    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 08:58:40.258618 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:40.258576    2574 projected.go:196] Error preparing data for projected volume kube-api-access-6hmx8 for pod openshift-network-diagnostics/network-check-target-sgxq8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:58:40.258699 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:40.258659    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8 podName:e8f68cb8-c496-4c1e-b79d-081e5fc8d121 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:41.258637022 +0000 UTC m=+4.222553424 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6hmx8" (UniqueName: "kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8") pod "network-check-target-sgxq8" (UID: "e8f68cb8-c496-4c1e-b79d-081e5fc8d121") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:58:40.482259 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:40.482222    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 08:53:40 +0000 UTC" deadline="2028-05-17 17:47:50.192295952 +0000 UTC"
Sep 07 08:58:40.482259 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:40.482252    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14840h49m9.71004724s"
Sep 07 08:58:40.535887 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:40.535857    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:40.536086 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:40.535978    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:58:40.837568 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:40.837533    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4df05be_d0fe_470e_bd44_7f506c6697b1.slice/crio-902ff5a1edecae87aca1f7ff816db5fb48514b0b80292cc668095ff27a471c1c WatchSource:0}: Error finding container 902ff5a1edecae87aca1f7ff816db5fb48514b0b80292cc668095ff27a471c1c: Status 404 returned error can't find the container with id 902ff5a1edecae87aca1f7ff816db5fb48514b0b80292cc668095ff27a471c1c
Sep 07 08:58:40.848787 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:40.847925    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40ddcfef_57f6_46fd_8d22_d4d0b5da9647.slice/crio-ceb9c1de12f740bbeea843977395ea786e63c0e86fca148307ff2b6f25f1e1fb WatchSource:0}: Error finding container ceb9c1de12f740bbeea843977395ea786e63c0e86fca148307ff2b6f25f1e1fb: Status 404 returned error can't find the container with id ceb9c1de12f740bbeea843977395ea786e63c0e86fca148307ff2b6f25f1e1fb
Sep 07 08:58:40.849848 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:40.849766    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa333c43_6b4b_4023_8665_bd82f6943f20.slice/crio-6407e47edb5ce5547e9e6c73008ee4ad510eeccd5320171b5ae234df40a1d88e WatchSource:0}: Error finding container 6407e47edb5ce5547e9e6c73008ee4ad510eeccd5320171b5ae234df40a1d88e: Status 404 returned error can't find the container with id 6407e47edb5ce5547e9e6c73008ee4ad510eeccd5320171b5ae234df40a1d88e
Sep 07 08:58:40.851754 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:58:40.850222    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23a08429_ee73_45ad_8a29_bbcf1a359f02.slice/crio-b1dac96eaac9720ddc52a9f5d9ff4f49c516db62366a56efc9946362e2e366ca WatchSource:0}: Error finding container b1dac96eaac9720ddc52a9f5d9ff4f49c516db62366a56efc9946362e2e366ca: Status 404 returned error can't find the container with id b1dac96eaac9720ddc52a9f5d9ff4f49c516db62366a56efc9946362e2e366ca
Sep 07 08:58:41.064401 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.064190    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs\") pod \"network-metrics-daemon-c594r\" (UID: \"2aee0981-afb8-47c7-8484-142812fc6d42\") " pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:41.064567 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:41.064343    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:58:41.064642 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:41.064566    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs podName:2aee0981-afb8-47c7-8484-142812fc6d42 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:43.064537005 +0000 UTC m=+6.028453208 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs") pod "network-metrics-daemon-c594r" (UID: "2aee0981-afb8-47c7-8484-142812fc6d42") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:58:41.165092 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.165062    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:41.165192 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:41.165167    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 08:58:41.165240 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:41.165215    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret podName:e3bc99a4-33c4-4a42-a55c-ef8de46f9502 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:43.165203027 +0000 UTC m=+6.129119228 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret") pod "global-pull-secret-syncer-z54bd" (UID: "e3bc99a4-33c4-4a42-a55c-ef8de46f9502") : object "kube-system"/"original-pull-secret" not registered
Sep 07 08:58:41.266013 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.265976    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hmx8\" (UniqueName: \"kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8\") pod \"network-check-target-sgxq8\" (UID: \"e8f68cb8-c496-4c1e-b79d-081e5fc8d121\") " pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:41.266211 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:41.266188    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 08:58:41.266301 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:41.266218    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 08:58:41.266301 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:41.266233    2574 projected.go:196] Error preparing data for projected volume kube-api-access-6hmx8 for pod openshift-network-diagnostics/network-check-target-sgxq8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:58:41.266406 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:41.266302    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8 podName:e8f68cb8-c496-4c1e-b79d-081e5fc8d121 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:43.266280993 +0000 UTC m=+6.230197198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6hmx8" (UniqueName: "kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8") pod "network-check-target-sgxq8" (UID: "e8f68cb8-c496-4c1e-b79d-081e5fc8d121") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:58:41.483418 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.483352    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 08:53:40 +0000 UTC" deadline="2028-03-15 10:17:42.056376253 +0000 UTC"
Sep 07 08:58:41.483418 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.483417    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13321h19m0.572963732s"
Sep 07 08:58:41.537624 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.536905    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:41.537624 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:41.537034    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:58:41.537624 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.537467    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:41.537624 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:41.537574    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:58:41.562494 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.562462    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m24pj" event={"ID":"5c88d609-ca70-4755-9e0d-b28846ba8ca0","Type":"ContainerStarted","Data":"2b7bcb57d5f86dcee6e1b91346a6ab1c5eb03e81a4c50d509332e690f2296f5e"}
Sep 07 08:58:41.576380 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.576353    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l8xfv" event={"ID":"23a08429-ee73-45ad-8a29-bbcf1a359f02","Type":"ContainerStarted","Data":"b1dac96eaac9720ddc52a9f5d9ff4f49c516db62366a56efc9946362e2e366ca"}
Sep 07 08:58:41.597200 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.597128    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4s9pv" event={"ID":"fa333c43-6b4b-4023-8665-bd82f6943f20","Type":"ContainerStarted","Data":"6407e47edb5ce5547e9e6c73008ee4ad510eeccd5320171b5ae234df40a1d88e"}
Sep 07 08:58:41.618108 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.618081    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-czgxz" event={"ID":"78b45f22-f23b-46a7-b3f6-21a960bf365e","Type":"ContainerStarted","Data":"80f75e805f58e96029dfdec444eff18518fe5492581b8f1c9af9ff663d7ef73d"}
Sep 07 08:58:41.628120 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.628091    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tjrhl" event={"ID":"99a03c01-5968-408a-a4ca-957acdce1bd7","Type":"ContainerStarted","Data":"bd9b611ce3ed1f8cea81ed40ce1480189f202a2922c86bebb2347b91a15b431e"}
Sep 07 08:58:41.635771 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.635692    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tfszw" event={"ID":"e4df05be-d0fe-470e-bd44-7f506c6697b1","Type":"ContainerStarted","Data":"902ff5a1edecae87aca1f7ff816db5fb48514b0b80292cc668095ff27a471c1c"}
Sep 07 08:58:41.641499 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.641467    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6" event={"ID":"90f4682b-cad1-4e6c-bda1-6e04cb6839cd","Type":"ContainerStarted","Data":"0b697b74bce576037b85dea95fe0144de0dcb2abff64a21a2b5d68d448ba52f7"}
Sep 07 08:58:41.649876 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.649850    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rpdbr" event={"ID":"40ddcfef-57f6-46fd-8d22-d4d0b5da9647","Type":"ContainerStarted","Data":"ceb9c1de12f740bbeea843977395ea786e63c0e86fca148307ff2b6f25f1e1fb"}
Sep 07 08:58:41.656504 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.656477    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw" event={"ID":"eebe8c71-1cd7-4040-af2d-7b636fd884c2","Type":"ContainerStarted","Data":"09ce9bbe7a56227ca87948774ad7a3d74df3eb74b5938a9e7c31f7a6678477a4"}
Sep 07 08:58:41.666538 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.666510    2574 generic.go:356] "Generic (PLEG): container finished" podID="501b893153a8107dd528995fa664b1c7" containerID="7ace1471ab527f4eb63ac177c25da2c2cd5d19be61fd3c460a04471e38b0fecb" exitCode=0
Sep 07 08:58:41.666637 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.666589    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal" event={"ID":"501b893153a8107dd528995fa664b1c7","Type":"ContainerDied","Data":"7ace1471ab527f4eb63ac177c25da2c2cd5d19be61fd3c460a04471e38b0fecb"}
Sep 07 08:58:41.685440 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.685416    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal" event={"ID":"9132b1637729b3158d4294075bfe2281","Type":"ContainerStarted","Data":"43acbd9cd7e4cadb09002f7e8f990deccf987e89b7ae9b7151552153f6733b24"}
Sep 07 08:58:41.699355 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:41.699173    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-81.ec2.internal" podStartSLOduration=2.699155181 podStartE2EDuration="2.699155181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:58:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:58:41.698288936 +0000 UTC m=+4.662205164" watchObservedRunningTime="2026-09-07 08:58:41.699155181 +0000 UTC m=+4.663071405"
Sep 07 08:58:42.535963 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:42.535931    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:42.536571 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:42.536071    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:58:42.696012 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:42.695938    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal" event={"ID":"501b893153a8107dd528995fa664b1c7","Type":"ContainerStarted","Data":"e3f2fd1267a6286551e13496b5deff65c2c87a3c8c78636bf08c3ed9b52d8d5a"}
Sep 07 08:58:43.081136 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:43.081105    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs\") pod \"network-metrics-daemon-c594r\" (UID: \"2aee0981-afb8-47c7-8484-142812fc6d42\") " pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:43.081342 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:43.081221    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:58:43.081342 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:43.081268    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs podName:2aee0981-afb8-47c7-8484-142812fc6d42 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:47.081254197 +0000 UTC m=+10.045170398 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs") pod "network-metrics-daemon-c594r" (UID: "2aee0981-afb8-47c7-8484-142812fc6d42") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:58:43.181683 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:43.181645    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:43.181878 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:43.181855    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 08:58:43.181957 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:43.181923    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret podName:e3bc99a4-33c4-4a42-a55c-ef8de46f9502 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:47.181904222 +0000 UTC m=+10.145820429 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret") pod "global-pull-secret-syncer-z54bd" (UID: "e3bc99a4-33c4-4a42-a55c-ef8de46f9502") : object "kube-system"/"original-pull-secret" not registered
Sep 07 08:58:43.282332 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:43.282290    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hmx8\" (UniqueName: \"kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8\") pod \"network-check-target-sgxq8\" (UID: \"e8f68cb8-c496-4c1e-b79d-081e5fc8d121\") " pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:43.282536 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:43.282512    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 08:58:43.282620 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:43.282577    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 08:58:43.282620 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:43.282591    2574 projected.go:196] Error preparing data for projected volume kube-api-access-6hmx8 for pod openshift-network-diagnostics/network-check-target-sgxq8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:58:43.282755 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:43.282658    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8 podName:e8f68cb8-c496-4c1e-b79d-081e5fc8d121 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:47.282637246 +0000 UTC m=+10.246553465 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6hmx8" (UniqueName: "kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8") pod "network-check-target-sgxq8" (UID: "e8f68cb8-c496-4c1e-b79d-081e5fc8d121") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:58:43.538511 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:43.538471    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:43.538985 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:43.538622    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:58:43.539376 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:43.539356    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:43.539496 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:43.539474    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:58:44.535606 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:44.535571    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:44.535861 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:44.535724    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:58:45.536498 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:45.536470    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:45.536995 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:45.536534    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:45.536995 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:45.536620    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:58:45.539810 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:45.539064    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:58:46.536206 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:46.536171    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:46.536398 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:46.536313    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:58:47.115687 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:47.115647    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs\") pod \"network-metrics-daemon-c594r\" (UID: \"2aee0981-afb8-47c7-8484-142812fc6d42\") " pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:47.116174 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:47.115839    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:58:47.116174 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:47.115913    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs podName:2aee0981-afb8-47c7-8484-142812fc6d42 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:55.115893017 +0000 UTC m=+18.079809218 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs") pod "network-metrics-daemon-c594r" (UID: "2aee0981-afb8-47c7-8484-142812fc6d42") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:58:47.216052 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:47.216011    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:47.216237 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:47.216167    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 08:58:47.216237 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:47.216222    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret podName:e3bc99a4-33c4-4a42-a55c-ef8de46f9502 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:55.216208088 +0000 UTC m=+18.180124288 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret") pod "global-pull-secret-syncer-z54bd" (UID: "e3bc99a4-33c4-4a42-a55c-ef8de46f9502") : object "kube-system"/"original-pull-secret" not registered
Sep 07 08:58:47.316589 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:47.316552    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hmx8\" (UniqueName: \"kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8\") pod \"network-check-target-sgxq8\" (UID: \"e8f68cb8-c496-4c1e-b79d-081e5fc8d121\") " pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:47.316796 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:47.316774    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 08:58:47.316873 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:47.316807    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 08:58:47.316873 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:47.316821    2574 projected.go:196] Error preparing data for projected volume kube-api-access-6hmx8 for pod openshift-network-diagnostics/network-check-target-sgxq8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:58:47.316958 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:47.316886    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8 podName:e8f68cb8-c496-4c1e-b79d-081e5fc8d121 nodeName:}" failed. No retries permitted until 2026-09-07 08:58:55.316865592 +0000 UTC m=+18.280781809 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6hmx8" (UniqueName: "kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8") pod "network-check-target-sgxq8" (UID: "e8f68cb8-c496-4c1e-b79d-081e5fc8d121") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:58:47.536014 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:47.535981    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:47.536176 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:47.536109    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:58:47.536411 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:47.536393    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:47.536513 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:47.536466    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:58:48.536511 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:48.536472    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:48.537141 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:48.536618    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:58:49.535781 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:49.535748    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:49.535958 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:49.535892    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:58:49.536683 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:49.536661    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:49.537058 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:49.536783    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:58:50.536050 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:50.536021    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:50.536211 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:50.536145    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:58:51.536247 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:51.536215    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:51.536652 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:51.536261    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:51.536652 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:51.536366    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:58:51.536652 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:51.536506    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:58:52.536439 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:52.536412    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:52.536846 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:52.536514    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:58:53.535903 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:53.535867    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:53.535903 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:53.535893    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:53.536126 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:53.535987    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:58:53.536126 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:53.536085    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:58:54.536057 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:54.536022    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:54.536454 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:54.536160    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:58:55.174438 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:55.174407    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs\") pod \"network-metrics-daemon-c594r\" (UID: \"2aee0981-afb8-47c7-8484-142812fc6d42\") " pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:55.174587 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:55.174522    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:58:55.174587 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:55.174575    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs podName:2aee0981-afb8-47c7-8484-142812fc6d42 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:11.174560497 +0000 UTC m=+34.138476696 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs") pod "network-metrics-daemon-c594r" (UID: "2aee0981-afb8-47c7-8484-142812fc6d42") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:58:55.275657 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:55.275621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:55.275808 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:55.275765    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 08:58:55.275843 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:55.275819    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret podName:e3bc99a4-33c4-4a42-a55c-ef8de46f9502 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:11.275806999 +0000 UTC m=+34.239723198 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret") pod "global-pull-secret-syncer-z54bd" (UID: "e3bc99a4-33c4-4a42-a55c-ef8de46f9502") : object "kube-system"/"original-pull-secret" not registered
Sep 07 08:58:55.376300 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:55.376270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hmx8\" (UniqueName: \"kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8\") pod \"network-check-target-sgxq8\" (UID: \"e8f68cb8-c496-4c1e-b79d-081e5fc8d121\") " pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:55.376446 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:55.376394    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 08:58:55.376446 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:55.376407    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 08:58:55.376446 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:55.376417    2574 projected.go:196] Error preparing data for projected volume kube-api-access-6hmx8 for pod openshift-network-diagnostics/network-check-target-sgxq8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:58:55.376536 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:55.376465    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8 podName:e8f68cb8-c496-4c1e-b79d-081e5fc8d121 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:11.376454007 +0000 UTC m=+34.340370206 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6hmx8" (UniqueName: "kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8") pod "network-check-target-sgxq8" (UID: "e8f68cb8-c496-4c1e-b79d-081e5fc8d121") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:58:55.536506 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:55.536490    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:55.536793 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:55.536573    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:58:55.536793 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:55.536655    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:55.536879 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:55.536790    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:58:56.536077 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:56.536040    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:56.536279 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:56.536168    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:58:57.536910 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:57.536880    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:57.537313 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:57.536987    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:58:57.537313 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:57.537033    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:57.537313 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:57.537140    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:58:58.535691 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:58.535661    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:58:58.535874 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:58.535783    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:58:59.536510 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.536332    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:58:59.536908 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.536344    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:58:59.536908 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:59.536593    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:58:59.536908 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:58:59.536742    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:58:59.737025 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.736951    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m24pj" event={"ID":"5c88d609-ca70-4755-9e0d-b28846ba8ca0","Type":"ContainerStarted","Data":"01286fa19f73ef9a9e09e8efcd0977d00d9a949f109a28d5d58a6b1a2e573185"}
Sep 07 08:58:59.738643 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.738620    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l8xfv" event={"ID":"23a08429-ee73-45ad-8a29-bbcf1a359f02","Type":"ContainerStarted","Data":"fabf6098906a3460bd7fa22241c11669881d0c572bd9192a2eddfc06100bf6f4"}
Sep 07 08:58:59.740128 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.740105    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4s9pv" event={"ID":"fa333c43-6b4b-4023-8665-bd82f6943f20","Type":"ContainerStarted","Data":"e4c5755c0d0cab501234f9c418c83405ca226b960ba882800ef7225be693e150"}
Sep 07 08:58:59.741593 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.741541    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-czgxz" event={"ID":"78b45f22-f23b-46a7-b3f6-21a960bf365e","Type":"ContainerStarted","Data":"24ad6f5454c8d7859ddc093bac7e6aa5e66b0826f1928fa86baeaa6ea3e82c09"}
Sep 07 08:58:59.743029 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.743004    2574 generic.go:356] "Generic (PLEG): container finished" podID="99a03c01-5968-408a-a4ca-957acdce1bd7" containerID="ad5f36e94d89a1ffbf323c102a090b28219547be2e33d6e58e1772fe9fb0b337" exitCode=0
Sep 07 08:58:59.743107 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.743073    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tjrhl" event={"ID":"99a03c01-5968-408a-a4ca-957acdce1bd7","Type":"ContainerDied","Data":"ad5f36e94d89a1ffbf323c102a090b28219547be2e33d6e58e1772fe9fb0b337"}
Sep 07 08:58:59.744466 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.744447    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6" event={"ID":"90f4682b-cad1-4e6c-bda1-6e04cb6839cd","Type":"ContainerStarted","Data":"30fad80c3c706a3b148a6ee523c6422b1683a621d26daba07bcde63b36cf292f"}
Sep 07 08:58:59.745763 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.745736    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rpdbr" event={"ID":"40ddcfef-57f6-46fd-8d22-d4d0b5da9647","Type":"ContainerStarted","Data":"109b4509661514c3fc7b258a00468eedaee872122034cb636494a4278245e88e"}
Sep 07 08:58:59.747269 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.747252    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 08:58:59.747596 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.747572    2574 generic.go:356] "Generic (PLEG): container finished" podID="eebe8c71-1cd7-4040-af2d-7b636fd884c2" containerID="a44fc83b271a2bf8b7fbdd3b58c557b86b65a98e01ca60b7964f6ecd2aadaf19" exitCode=1
Sep 07 08:58:59.747653 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.747607    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw" event={"ID":"eebe8c71-1cd7-4040-af2d-7b636fd884c2","Type":"ContainerDied","Data":"a44fc83b271a2bf8b7fbdd3b58c557b86b65a98e01ca60b7964f6ecd2aadaf19"}
Sep 07 08:58:59.747691 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.747674    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw" event={"ID":"eebe8c71-1cd7-4040-af2d-7b636fd884c2","Type":"ContainerStarted","Data":"37d3bb289db5cb94b6314f4990aff1f23f3eb4b3206a4a7652a3802ea72e9b05"}
Sep 07 08:58:59.751764 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.751681    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m24pj" podStartSLOduration=3.547452637 podStartE2EDuration="21.75167053s" totalImagesPullingTime="18.204217893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:58:59.75138572 +0000 UTC m=+22.715301941" watchObservedRunningTime="2026-09-07 08:58:59.75167053 +0000 UTC m=+22.715586780"
Sep 07 08:58:59.752007 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.751985    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-81.ec2.internal" podStartSLOduration=20.751979073 podStartE2EDuration="20.751979073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:58:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:58:42.710812385 +0000 UTC m=+5.674728624" watchObservedRunningTime="2026-09-07 08:58:59.751979073 +0000 UTC m=+22.715895295"
Sep 07 08:58:59.790770 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.790733    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rpdbr" podStartSLOduration=3.587026805 podStartE2EDuration="21.79069792s" totalImagesPullingTime="18.203671115s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:58:59.764838605 +0000 UTC m=+22.728754859" watchObservedRunningTime="2026-09-07 08:58:59.79069792 +0000 UTC m=+22.754614141"
Sep 07 08:58:59.813866 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.813831    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-czgxz" podStartSLOduration=3.671806283 podStartE2EDuration="21.813819357s" totalImagesPullingTime="18.142013074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:58:59.813376179 +0000 UTC m=+22.777292399" watchObservedRunningTime="2026-09-07 08:58:59.813819357 +0000 UTC m=+22.777735579"
Sep 07 08:58:59.841020 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.840876    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4s9pv" podStartSLOduration=3.64428838 podStartE2EDuration="21.84086495s" totalImagesPullingTime="18.19657657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:58:59.82617884 +0000 UTC m=+22.790095060" watchObservedRunningTime="2026-09-07 08:58:59.84086495 +0000 UTC m=+22.804781171"
Sep 07 08:58:59.841109 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:58:59.841085    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-l8xfv" podStartSLOduration=3.631662079 podStartE2EDuration="21.841076594s" totalImagesPullingTime="18.209414515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:58:59.84065813 +0000 UTC m=+22.804574348" watchObservedRunningTime="2026-09-07 08:58:59.841076594 +0000 UTC m=+22.804992816"
Sep 07 08:59:00.514122 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:00.514100    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 07 08:59:00.536552 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:00.536529    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:59:00.537180 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:00.536684    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:59:00.752429 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:00.752349    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6" event={"ID":"90f4682b-cad1-4e6c-bda1-6e04cb6839cd","Type":"ContainerStarted","Data":"044401442b9f88e3f0d7a3cc5c7df1db8a6d2719efc968a207cbe93b90422ca8"}
Sep 07 08:59:00.755616 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:00.755589    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 08:59:00.756075 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:00.756050    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw" event={"ID":"eebe8c71-1cd7-4040-af2d-7b636fd884c2","Type":"ContainerStarted","Data":"5c9d943bab94365664d8ba3e1698da5210c1d706cba9e5a63af34a1ecece6286"}
Sep 07 08:59:00.756209 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:00.756131    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw" event={"ID":"eebe8c71-1cd7-4040-af2d-7b636fd884c2","Type":"ContainerStarted","Data":"b00cf0c442c2fb8765121649468b72f7cac50ff4b9f07efec98a6ea21f5f537f"}
Sep 07 08:59:00.756209 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:00.756166    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw" event={"ID":"eebe8c71-1cd7-4040-af2d-7b636fd884c2","Type":"ContainerStarted","Data":"6953be777cce68d0cb8468a20e334c35a2b4ba3e58d76d157b2bc3aa7bd84bde"}
Sep 07 08:59:00.756209 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:00.756192    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw" event={"ID":"eebe8c71-1cd7-4040-af2d-7b636fd884c2","Type":"ContainerStarted","Data":"186456733aa6596024a4d0cade1d00558f342abe7b82d70b58bb00b2006391fa"}
Sep 07 08:59:00.757660 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:00.757611    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tfszw" event={"ID":"e4df05be-d0fe-470e-bd44-7f506c6697b1","Type":"ContainerStarted","Data":"b4c6608c5422f7be8b711499c9de6a6a869e5767009581d98ede70a17afa8599"}
Sep 07 08:59:01.514574 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:01.514466    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-07T08:59:00.514116791Z","UUID":"f2c3c8bc-ef10-4e44-a051-a8bbb96b3375","Handler":null,"Name":"","Endpoint":""}
Sep 07 08:59:01.516036 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:01.516015    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 07 08:59:01.516135 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:01.516051    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 07 08:59:01.535960 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:01.535936    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:59:01.536096 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:01.536070    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:59:01.536096 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:01.536109    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:59:01.536273 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:01.536181    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:59:01.797683 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:01.797617    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-czgxz"
Sep 07 08:59:01.798286 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:01.798269    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-czgxz"
Sep 07 08:59:01.810623 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:01.810587    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tfszw" podStartSLOduration=5.628322852 podStartE2EDuration="23.810570796s" totalImagesPullingTime="18.182247944s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:00.770767307 +0000 UTC m=+23.734683554" watchObservedRunningTime="2026-09-07 08:59:01.810570796 +0000 UTC m=+24.774487018"
Sep 07 08:59:02.535633 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:02.535560    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:59:02.535816 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:02.535673    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:59:02.767793 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:02.767747    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6" event={"ID":"90f4682b-cad1-4e6c-bda1-6e04cb6839cd","Type":"ContainerStarted","Data":"eaeef2d93258d78de265baa9adbfa2e3b5107da1a83681a2d5937f454b9a0ba1"}
Sep 07 08:59:02.771591 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:02.771564    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 08:59:02.772082 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:02.772056    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw" event={"ID":"eebe8c71-1cd7-4040-af2d-7b636fd884c2","Type":"ContainerStarted","Data":"2bb8bd40fb01742e44b9eb2a3c292a99f562ff913b7831286f939a5f293a9dff"}
Sep 07 08:59:02.772344 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:02.772321    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-czgxz"
Sep 07 08:59:02.772863 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:02.772844    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-czgxz"
Sep 07 08:59:02.784054 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:02.784013    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7ckm6" podStartSLOduration=4.462915593 podStartE2EDuration="24.784000708s" totalImagesPullingTime="20.321085115s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:58:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:02.78372698 +0000 UTC m=+25.747643216" watchObservedRunningTime="2026-09-07 08:59:02.784000708 +0000 UTC m=+25.747916929"
Sep 07 08:59:03.535654 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:03.535612    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:59:03.536134 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:03.535772    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:59:03.536200 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:03.536158    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:59:03.536306 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:03.536269    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:59:04.536690 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:04.536517    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:59:04.537117 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:04.536774    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:59:04.778277 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:04.778193    2574 generic.go:356] "Generic (PLEG): container finished" podID="99a03c01-5968-408a-a4ca-957acdce1bd7" containerID="8de12902d90e6e05cbd21f05c8015602f4cafd8df4cdd4be7fe937dd95fdb725" exitCode=0
Sep 07 08:59:04.778431 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:04.778281    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tjrhl" event={"ID":"99a03c01-5968-408a-a4ca-957acdce1bd7","Type":"ContainerDied","Data":"8de12902d90e6e05cbd21f05c8015602f4cafd8df4cdd4be7fe937dd95fdb725"}
Sep 07 08:59:04.781930 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:04.781914    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 08:59:04.782355 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:04.782333    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw" event={"ID":"eebe8c71-1cd7-4040-af2d-7b636fd884c2","Type":"ContainerStarted","Data":"755e54082d0d57f95d364e491a0862c8f69508b10815cac53da70d313693e912"}
Sep 07 08:59:04.782618 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:04.782598    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:59:04.782684 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:04.782633    2574 scope.go:117] "RemoveContainer" containerID="a44fc83b271a2bf8b7fbdd3b58c557b86b65a98e01ca60b7964f6ecd2aadaf19"
Sep 07 08:59:04.782684 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:04.782660    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:59:04.800769 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:04.800752    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:59:05.536296 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:05.536136    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:59:05.536393 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:05.536168    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:59:05.536430 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:05.536390    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:59:05.536544 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:05.536524    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:59:05.788135 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:05.788073    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 08:59:05.788586 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:05.788542    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw" event={"ID":"eebe8c71-1cd7-4040-af2d-7b636fd884c2","Type":"ContainerStarted","Data":"bfab2f9b83af97f79e4a5ea190407144db5d41d96d2fcdd0d36c862204438dc2"}
Sep 07 08:59:05.788870 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:05.788845    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:59:05.791060 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:05.791036    2574 generic.go:356] "Generic (PLEG): container finished" podID="99a03c01-5968-408a-a4ca-957acdce1bd7" containerID="c37cecd1f5158b981678b3d81e16d53472d361c332214ad14e99100c8246e024" exitCode=0
Sep 07 08:59:05.791181 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:05.791070    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tjrhl" event={"ID":"99a03c01-5968-408a-a4ca-957acdce1bd7","Type":"ContainerDied","Data":"c37cecd1f5158b981678b3d81e16d53472d361c332214ad14e99100c8246e024"}
Sep 07 08:59:05.803941 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:05.803924    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:59:05.812674 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:05.812636    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw" podStartSLOduration=9.233869668 podStartE2EDuration="27.812619838s" totalImagesPullingTime="18.57875017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:05.811523281 +0000 UTC m=+28.775439501" watchObservedRunningTime="2026-09-07 08:59:05.812619838 +0000 UTC m=+28.776536060"
Sep 07 08:59:06.244502 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:06.244467    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z54bd"]
Sep 07 08:59:06.244664 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:06.244622    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:59:06.244766 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:06.244749    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:59:06.246871 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:06.246847    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c594r"]
Sep 07 08:59:06.246965 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:06.246937    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:59:06.247058 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:06.247030    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:59:06.259063 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:06.259039    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sgxq8"]
Sep 07 08:59:06.259168 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:06.259114    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:59:06.259222 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:06.259172    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:59:06.795345 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:06.795314    2574 generic.go:356] "Generic (PLEG): container finished" podID="99a03c01-5968-408a-a4ca-957acdce1bd7" containerID="6b0fcc4714821f28251824e6030e869f3552c2e390cf79df87a1d0c0e6eae407" exitCode=0
Sep 07 08:59:06.795867 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:06.795402    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tjrhl" event={"ID":"99a03c01-5968-408a-a4ca-957acdce1bd7","Type":"ContainerDied","Data":"6b0fcc4714821f28251824e6030e869f3552c2e390cf79df87a1d0c0e6eae407"}
Sep 07 08:59:07.537016 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:07.536991    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:59:07.537186 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:07.537074    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:59:07.537186 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:07.537122    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:59:07.537186 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:07.537144    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:59:07.537350 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:07.537191    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:59:07.537350 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:07.537270    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:59:09.536499 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:09.536300    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:59:09.537110 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:09.536300    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:59:09.537110 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:09.536329    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:59:09.537110 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:09.536752    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:59:09.537110 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:09.536605    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:59:09.537110 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:09.536824    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:59:11.193385 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:11.193182    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs\") pod \"network-metrics-daemon-c594r\" (UID: \"2aee0981-afb8-47c7-8484-142812fc6d42\") " pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:59:11.193385 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:11.193315    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:11.193385 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:11.193369    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs podName:2aee0981-afb8-47c7-8484-142812fc6d42 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:43.193354629 +0000 UTC m=+66.157270848 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs") pod "network-metrics-daemon-c594r" (UID: "2aee0981-afb8-47c7-8484-142812fc6d42") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 08:59:11.294066 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:11.294040    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:59:11.294181 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:11.294159    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:11.294220 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:11.294203    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret podName:e3bc99a4-33c4-4a42-a55c-ef8de46f9502 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:43.294191383 +0000 UTC m=+66.258107582 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret") pod "global-pull-secret-syncer-z54bd" (UID: "e3bc99a4-33c4-4a42-a55c-ef8de46f9502") : object "kube-system"/"original-pull-secret" not registered
Sep 07 08:59:11.395334 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:11.395280    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hmx8\" (UniqueName: \"kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8\") pod \"network-check-target-sgxq8\" (UID: \"e8f68cb8-c496-4c1e-b79d-081e5fc8d121\") " pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:59:11.395451 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:11.395437    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 08:59:11.395498 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:11.395456    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 08:59:11.395498 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:11.395466    2574 projected.go:196] Error preparing data for projected volume kube-api-access-6hmx8 for pod openshift-network-diagnostics/network-check-target-sgxq8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:11.395567 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:11.395515    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8 podName:e8f68cb8-c496-4c1e-b79d-081e5fc8d121 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:43.395501525 +0000 UTC m=+66.359417723 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6hmx8" (UniqueName: "kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8") pod "network-check-target-sgxq8" (UID: "e8f68cb8-c496-4c1e-b79d-081e5fc8d121") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 08:59:11.535940 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:11.535912    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:59:11.536103 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:11.535911    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:59:11.536103 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:11.535996    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-z54bd" podUID="e3bc99a4-33c4-4a42-a55c-ef8de46f9502"
Sep 07 08:59:11.536103 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:11.536087    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c594r" podUID="2aee0981-afb8-47c7-8484-142812fc6d42"
Sep 07 08:59:11.536283 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:11.536133    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:59:11.536283 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:11.536194    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sgxq8" podUID="e8f68cb8-c496-4c1e-b79d-081e5fc8d121"
Sep 07 08:59:11.810944 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:11.810908    2574 generic.go:356] "Generic (PLEG): container finished" podID="99a03c01-5968-408a-a4ca-957acdce1bd7" containerID="7010edac4cbfd3c978db8665761c0397b1d88e3982200d2089459556f82f6028" exitCode=0
Sep 07 08:59:11.811075 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:11.810959    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tjrhl" event={"ID":"99a03c01-5968-408a-a4ca-957acdce1bd7","Type":"ContainerDied","Data":"7010edac4cbfd3c978db8665761c0397b1d88e3982200d2089459556f82f6028"}
Sep 07 08:59:12.343503 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.343478    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-81.ec2.internal" event="NodeReady"
Sep 07 08:59:12.343941 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.343577    2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 07 08:59:12.376278 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.376244    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c9bc4456f-xpnlz"]
Sep 07 08:59:12.395521 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.395492    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7g5t9"]
Sep 07 08:59:12.395644 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.395603    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.397592 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.397570    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 07 08:59:12.397729 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.397635    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 07 08:59:12.397917 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.397896    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 07 08:59:12.398064 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.398046    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8sncs\""
Sep 07 08:59:12.407644 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.407624    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 07 08:59:12.417653 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.417632    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c9bc4456f-xpnlz"]
Sep 07 08:59:12.417763 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.417658    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hzt78"]
Sep 07 08:59:12.417826 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.417788    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:12.419828 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.419804    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 07 08:59:12.419912 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.419807    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hmltn\""
Sep 07 08:59:12.419912 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.419812    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 07 08:59:12.435283 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.435265    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7g5t9"]
Sep 07 08:59:12.435357 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.435295    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hzt78"]
Sep 07 08:59:12.435394 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.435360    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hzt78"
Sep 07 08:59:12.437193 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.437177    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 07 08:59:12.437266 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.437239    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 07 08:59:12.437343 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.437312    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fc2n8\""
Sep 07 08:59:12.437448 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.437438    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 07 08:59:12.503580 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.503561    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d8372a4-648d-4544-b928-faa61f9b936a-ca-trust-extracted\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.503687 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.503602    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-bound-sa-token\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.503687 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.503619    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:12.503687 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.503633    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e215d11-e0ca-493d-a261-35932fbea87c-tmp-dir\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:12.503687 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.503678    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2e215d11-e0ca-493d-a261-35932fbea87c-config-volume\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:12.503833 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.503736    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d8372a4-648d-4544-b928-faa61f9b936a-registry-certificates\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.503833 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.503754    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9d7n\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-kube-api-access-d9d7n\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.503833 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.503771    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d8372a4-648d-4544-b928-faa61f9b936a-image-registry-private-configuration\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.503833 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.503786    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d8372a4-648d-4544-b928-faa61f9b936a-trusted-ca\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.503833 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.503802    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xprj\" (UniqueName: \"kubernetes.io/projected/2e215d11-e0ca-493d-a261-35932fbea87c-kube-api-access-7xprj\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:12.503986 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.503860    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.503986 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.503897    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d8372a4-648d-4544-b928-faa61f9b936a-installation-pull-secrets\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.604776 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.604707    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrdrj\" (UniqueName: \"kubernetes.io/projected/c82b73b1-7d84-4d8f-be08-c7cf655fe39d-kube-api-access-jrdrj\") pod \"ingress-canary-hzt78\" (UID: \"c82b73b1-7d84-4d8f-be08-c7cf655fe39d\") " pod="openshift-ingress-canary/ingress-canary-hzt78"
Sep 07 08:59:12.604776 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.604752    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d8372a4-648d-4544-b928-faa61f9b936a-registry-certificates\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.604776 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.604770    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9d7n\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-kube-api-access-d9d7n\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.605053 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.604797    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d8372a4-648d-4544-b928-faa61f9b936a-image-registry-private-configuration\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.605053 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.604820    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d8372a4-648d-4544-b928-faa61f9b936a-trusted-ca\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.605053 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.604839    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xprj\" (UniqueName: \"kubernetes.io/projected/2e215d11-e0ca-493d-a261-35932fbea87c-kube-api-access-7xprj\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:12.605053 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.604880    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.605053 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.604916    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d8372a4-648d-4544-b928-faa61f9b936a-installation-pull-secrets\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.605053 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.604943    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d8372a4-648d-4544-b928-faa61f9b936a-ca-trust-extracted\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.605053 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.604969    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c82b73b1-7d84-4d8f-be08-c7cf655fe39d-cert\") pod \"ingress-canary-hzt78\" (UID: \"c82b73b1-7d84-4d8f-be08-c7cf655fe39d\") " pod="openshift-ingress-canary/ingress-canary-hzt78"
Sep 07 08:59:12.605053 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.605016    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-bound-sa-token\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.605053 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:12.605026    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 08:59:12.605053 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:12.605043    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c9bc4456f-xpnlz: secret "image-registry-tls" not found
Sep 07 08:59:12.605502 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:12.605095    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls podName:5d8372a4-648d-4544-b928-faa61f9b936a nodeName:}" failed. No retries permitted until 2026-09-07 08:59:13.105076631 +0000 UTC m=+36.068992830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls") pod "image-registry-c9bc4456f-xpnlz" (UID: "5d8372a4-648d-4544-b928-faa61f9b936a") : secret "image-registry-tls" not found
Sep 07 08:59:12.605502 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:12.605106    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 08:59:12.605502 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:12.605164    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls podName:2e215d11-e0ca-493d-a261-35932fbea87c nodeName:}" failed. No retries permitted until 2026-09-07 08:59:13.105148369 +0000 UTC m=+36.069064589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls") pod "dns-default-7g5t9" (UID: "2e215d11-e0ca-493d-a261-35932fbea87c") : secret "dns-default-metrics-tls" not found
Sep 07 08:59:12.605502 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.605039    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:12.605502 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.605210    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e215d11-e0ca-493d-a261-35932fbea87c-tmp-dir\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:12.605502 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.605241    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2e215d11-e0ca-493d-a261-35932fbea87c-config-volume\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:12.605502 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.605352    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d8372a4-648d-4544-b928-faa61f9b936a-ca-trust-extracted\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.605502 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.605422    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d8372a4-648d-4544-b928-faa61f9b936a-registry-certificates\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.605920 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.605520    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e215d11-e0ca-493d-a261-35932fbea87c-tmp-dir\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:12.605920 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.605735    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2e215d11-e0ca-493d-a261-35932fbea87c-config-volume\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:12.606250 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.606228    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d8372a4-648d-4544-b928-faa61f9b936a-trusted-ca\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.608890 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.608870    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d8372a4-648d-4544-b928-faa61f9b936a-installation-pull-secrets\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.608977 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.608873    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d8372a4-648d-4544-b928-faa61f9b936a-image-registry-private-configuration\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.618816 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.618781    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xprj\" (UniqueName: \"kubernetes.io/projected/2e215d11-e0ca-493d-a261-35932fbea87c-kube-api-access-7xprj\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:12.618915 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.618896    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-bound-sa-token\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.620637 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.620618    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9d7n\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-kube-api-access-d9d7n\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:12.705962 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.705940    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrdrj\" (UniqueName: \"kubernetes.io/projected/c82b73b1-7d84-4d8f-be08-c7cf655fe39d-kube-api-access-jrdrj\") pod \"ingress-canary-hzt78\" (UID: \"c82b73b1-7d84-4d8f-be08-c7cf655fe39d\") " pod="openshift-ingress-canary/ingress-canary-hzt78"
Sep 07 08:59:12.706097 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.705998    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c82b73b1-7d84-4d8f-be08-c7cf655fe39d-cert\") pod \"ingress-canary-hzt78\" (UID: \"c82b73b1-7d84-4d8f-be08-c7cf655fe39d\") " pod="openshift-ingress-canary/ingress-canary-hzt78"
Sep 07 08:59:12.708019 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.708001    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c82b73b1-7d84-4d8f-be08-c7cf655fe39d-cert\") pod \"ingress-canary-hzt78\" (UID: \"c82b73b1-7d84-4d8f-be08-c7cf655fe39d\") " pod="openshift-ingress-canary/ingress-canary-hzt78"
Sep 07 08:59:12.713432 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.713415    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrdrj\" (UniqueName: \"kubernetes.io/projected/c82b73b1-7d84-4d8f-be08-c7cf655fe39d-kube-api-access-jrdrj\") pod \"ingress-canary-hzt78\" (UID: \"c82b73b1-7d84-4d8f-be08-c7cf655fe39d\") " pod="openshift-ingress-canary/ingress-canary-hzt78"
Sep 07 08:59:12.742489 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.742460    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hzt78"
Sep 07 08:59:12.816284 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.816250    2574 generic.go:356] "Generic (PLEG): container finished" podID="99a03c01-5968-408a-a4ca-957acdce1bd7" containerID="e4a3eb99bcea248aef249cfa2784e4145f9ec7cc1df3619c1474cce91936a097" exitCode=0
Sep 07 08:59:12.816415 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:12.816309    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tjrhl" event={"ID":"99a03c01-5968-408a-a4ca-957acdce1bd7","Type":"ContainerDied","Data":"e4a3eb99bcea248aef249cfa2784e4145f9ec7cc1df3619c1474cce91936a097"}
Sep 07 08:59:13.049872 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.049644    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hzt78"]
Sep 07 08:59:13.052843 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:13.052819    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc82b73b1_7d84_4d8f_be08_c7cf655fe39d.slice/crio-66b104a334f53b16f13f80354750470db34519ee96ed23294dc3d0efb8a8c0a4 WatchSource:0}: Error finding container 66b104a334f53b16f13f80354750470db34519ee96ed23294dc3d0efb8a8c0a4: Status 404 returned error can't find the container with id 66b104a334f53b16f13f80354750470db34519ee96ed23294dc3d0efb8a8c0a4
Sep 07 08:59:13.110087 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.110063    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:13.110179 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.110115    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:13.110218 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:13.110197    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 08:59:13.110218 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:13.110213    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c9bc4456f-xpnlz: secret "image-registry-tls" not found
Sep 07 08:59:13.110280 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:13.110252    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls podName:5d8372a4-648d-4544-b928-faa61f9b936a nodeName:}" failed. No retries permitted until 2026-09-07 08:59:14.110239019 +0000 UTC m=+37.074155217 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls") pod "image-registry-c9bc4456f-xpnlz" (UID: "5d8372a4-648d-4544-b928-faa61f9b936a") : secret "image-registry-tls" not found
Sep 07 08:59:13.110280 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:13.110200    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 08:59:13.110345 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:13.110310    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls podName:2e215d11-e0ca-493d-a261-35932fbea87c nodeName:}" failed. No retries permitted until 2026-09-07 08:59:14.110299728 +0000 UTC m=+37.074215927 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls") pod "dns-default-7g5t9" (UID: "2e215d11-e0ca-493d-a261-35932fbea87c") : secret "dns-default-metrics-tls" not found
Sep 07 08:59:13.536241 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.536200    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:59:13.536241 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.536228    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:59:13.536981 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.536430    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:59:13.538622 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.538602    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 08:59:13.538803 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.538610    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 08:59:13.539030 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.539010    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 07 08:59:13.539030 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.539027    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-txgtt\""
Sep 07 08:59:13.539267 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.539074    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mlzfg\""
Sep 07 08:59:13.539267 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.539010    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 07 08:59:13.820867 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.820776    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hzt78" event={"ID":"c82b73b1-7d84-4d8f-be08-c7cf655fe39d","Type":"ContainerStarted","Data":"66b104a334f53b16f13f80354750470db34519ee96ed23294dc3d0efb8a8c0a4"}
Sep 07 08:59:13.824660 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.824636    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tjrhl" event={"ID":"99a03c01-5968-408a-a4ca-957acdce1bd7","Type":"ContainerStarted","Data":"989253ede9afeee4044c87dbb54098c8b7dd043db4cb146e4b809850a5de43e6"}
Sep 07 08:59:13.844226 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.844176    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tjrhl" podStartSLOduration=7.651914323 podStartE2EDuration="35.844161305s" totalImagesPullingTime="28.192246982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:58:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:13.843023351 +0000 UTC m=+36.806939598" watchObservedRunningTime="2026-09-07 08:59:13.844161305 +0000 UTC m=+36.808077525"
Sep 07 08:59:13.988010 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:13.987966    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:14.066744 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:14.066697    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:14.117809 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:14.117733    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:14.117809 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:14.117794    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:14.118033 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:14.117890    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 08:59:14.118033 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:14.117955    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls podName:2e215d11-e0ca-493d-a261-35932fbea87c nodeName:}" failed. No retries permitted until 2026-09-07 08:59:16.117936807 +0000 UTC m=+39.081853009 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls") pod "dns-default-7g5t9" (UID: "2e215d11-e0ca-493d-a261-35932fbea87c") : secret "dns-default-metrics-tls" not found
Sep 07 08:59:14.118033 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:14.117895    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 08:59:14.118033 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:14.117979    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c9bc4456f-xpnlz: secret "image-registry-tls" not found
Sep 07 08:59:14.118033 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:14.118017    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls podName:5d8372a4-648d-4544-b928-faa61f9b936a nodeName:}" failed. No retries permitted until 2026-09-07 08:59:16.118006864 +0000 UTC m=+39.081923077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls") pod "image-registry-c9bc4456f-xpnlz" (UID: "5d8372a4-648d-4544-b928-faa61f9b936a") : secret "image-registry-tls" not found
Sep 07 08:59:14.711999 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:14.711953    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:14.769390 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:14.769337    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:15.831356 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:15.831324    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hzt78" event={"ID":"c82b73b1-7d84-4d8f-be08-c7cf655fe39d","Type":"ContainerStarted","Data":"3fc6ce38dd841d703fccbd32c80c6d7281be98ec7f2ba4840fbc6c3816214681"}
Sep 07 08:59:15.845309 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:15.845262    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hzt78" podStartSLOduration=1.5044218520000001 podStartE2EDuration="3.845245832s" totalImagesPullingTime="2.34082398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:15.844750114 +0000 UTC m=+38.808666355" watchObservedRunningTime="2026-09-07 08:59:15.845245832 +0000 UTC m=+38.809162052"
Sep 07 08:59:16.132606 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:16.132538    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:16.132791 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:16.132610    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:16.132791 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:16.132691    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 08:59:16.132791 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:16.132691    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 08:59:16.132791 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:16.132704    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c9bc4456f-xpnlz: secret "image-registry-tls" not found
Sep 07 08:59:16.132791 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:16.132773    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls podName:2e215d11-e0ca-493d-a261-35932fbea87c nodeName:}" failed. No retries permitted until 2026-09-07 08:59:20.132755136 +0000 UTC m=+43.096671335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls") pod "dns-default-7g5t9" (UID: "2e215d11-e0ca-493d-a261-35932fbea87c") : secret "dns-default-metrics-tls" not found
Sep 07 08:59:16.132791 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:16.132786    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls podName:5d8372a4-648d-4544-b928-faa61f9b936a nodeName:}" failed. No retries permitted until 2026-09-07 08:59:20.132780861 +0000 UTC m=+43.096697060 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls") pod "image-registry-c9bc4456f-xpnlz" (UID: "5d8372a4-648d-4544-b928-faa61f9b936a") : secret "image-registry-tls" not found
Sep 07 08:59:17.652698 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:17.652650    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:17.701921 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:17.701885    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:18.772000 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.771970    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-6ds2f"]
Sep 07 08:59:18.798048 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.798023    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4"]
Sep 07 08:59:18.798190 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.798129    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-6ds2f"
Sep 07 08:59:18.801804 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.801781    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-5r2r9\""
Sep 07 08:59:18.813862 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.813844    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"]
Sep 07 08:59:18.813963 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.813948    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4"
Sep 07 08:59:18.817117 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.817098    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 07 08:59:18.817212 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.817100    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 07 08:59:18.817302 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.817285    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-llc9f\""
Sep 07 08:59:18.817633 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.817621    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 07 08:59:18.822519 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.822505    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 07 08:59:18.836227 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.836209    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-hm7ht"]
Sep 07 08:59:18.836343 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.836329    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:18.841274 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.841247    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 07 08:59:18.842079 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.842055    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 07 08:59:18.842189 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.842058    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 07 08:59:18.842189 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.842140    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-65k9n\""
Sep 07 08:59:18.842301 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.842216    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 07 08:59:18.860041 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.860023    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-4ggpx"]
Sep 07 08:59:18.860144 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.860132    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:18.862040 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.862020    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 07 08:59:18.862126 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.862091    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 07 08:59:18.862502 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.862486    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 07 08:59:18.862558 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.862508    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 07 08:59:18.862597 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.862580    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-rjhpc\""
Sep 07 08:59:18.867490 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.867469    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 07 08:59:18.878302 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.878164    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-6ds2f"]
Sep 07 08:59:18.878302 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.878198    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4"]
Sep 07 08:59:18.878302 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.878218    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"]
Sep 07 08:59:18.878302 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.878235    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-4ggpx"]
Sep 07 08:59:18.878302 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.878242    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:18.878302 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.878251    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-hm7ht"]
Sep 07 08:59:18.879221 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.879203    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-dbgqp"]
Sep 07 08:59:18.880155 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.880138    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 07 08:59:18.880265 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.880248    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-v7h9k\""
Sep 07 08:59:18.880359 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.880341    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 07 08:59:18.880765 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.880602    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 07 08:59:18.880765 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.880604    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 07 08:59:18.888208 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.888183    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754"]
Sep 07 08:59:18.888307 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.888294    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp"
Sep 07 08:59:18.890179 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.890149    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 07 08:59:18.890746 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.890728    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9fp47\""
Sep 07 08:59:18.891359 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.891342    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 07 08:59:18.891463 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.891427    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 07 08:59:18.902474 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.902454    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"]
Sep 07 08:59:18.902574 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.902559    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754"
Sep 07 08:59:18.904775 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.904755    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 07 08:59:18.904874 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.904818    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 07 08:59:18.904942 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.904881    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 07 08:59:18.904942 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.904914    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 07 08:59:18.905049 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.905014    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-mjpth\""
Sep 07 08:59:18.916450 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.916433    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-9znqs"]
Sep 07 08:59:18.916555 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.916543    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"
Sep 07 08:59:18.918363 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.918347    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 07 08:59:18.918363 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.918352    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-kf7fc\""
Sep 07 08:59:18.918569 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.918556    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 07 08:59:18.918611 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.918582    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 07 08:59:18.928598 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.928582    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-54ff77667d-jgqz5"]
Sep 07 08:59:18.928723 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.928697    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-9znqs"
Sep 07 08:59:18.930587 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.930571    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 07 08:59:18.930587 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.930586    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-nhmpf\""
Sep 07 08:59:18.930754 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.930634    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 07 08:59:18.940728 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.940699    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754"]
Sep 07 08:59:18.940805 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.940738    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-9znqs"]
Sep 07 08:59:18.940805 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.940754    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-dbgqp"]
Sep 07 08:59:18.940805 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.940768    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54ff77667d-jgqz5"]
Sep 07 08:59:18.940805 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.940783    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"]
Sep 07 08:59:18.940805 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.940783    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:18.942502 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.942486    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 07 08:59:18.942693 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.942678    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 07 08:59:18.942784 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.942772    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 07 08:59:18.942928 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.942908    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 07 08:59:18.943022 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.942943    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 07 08:59:18.943022 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.942952    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xz9dr\""
Sep 07 08:59:18.943149 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.943054    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 07 08:59:18.951145 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.951127    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:18.951229 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.951154    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f96ca98e-8ab7-4335-b49f-c97435d6eab7-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cmlg4\" (UID: \"f96ca98e-8ab7-4335-b49f-c97435d6eab7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4"
Sep 07 08:59:18.951229 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.951175    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f96ca98e-8ab7-4335-b49f-c97435d6eab7-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cmlg4\" (UID: \"f96ca98e-8ab7-4335-b49f-c97435d6eab7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4"
Sep 07 08:59:18.951314 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.951249    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f4d4c13f-bc7f-4f46-b32b-906e6136122b-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:18.951314 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.951281    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppm6s\" (UniqueName: \"kubernetes.io/projected/f4d4c13f-bc7f-4f46-b32b-906e6136122b-kube-api-access-ppm6s\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:18.951314 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.951301    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b61be2b6-3472-4523-95d6-c964c440fe56-config\") pod \"console-operator-85b454874b-hm7ht\" (UID: \"b61be2b6-3472-4523-95d6-c964c440fe56\") " pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:18.951456 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.951319    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgg4l\" (UniqueName: \"kubernetes.io/projected/b61be2b6-3472-4523-95d6-c964c440fe56-kube-api-access-bgg4l\") pod \"console-operator-85b454874b-hm7ht\" (UID: \"b61be2b6-3472-4523-95d6-c964c440fe56\") " pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:18.951456 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.951351    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b61be2b6-3472-4523-95d6-c964c440fe56-serving-cert\") pod \"console-operator-85b454874b-hm7ht\" (UID: \"b61be2b6-3472-4523-95d6-c964c440fe56\") " pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:18.951456 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.951369    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b61be2b6-3472-4523-95d6-c964c440fe56-trusted-ca\") pod \"console-operator-85b454874b-hm7ht\" (UID: \"b61be2b6-3472-4523-95d6-c964c440fe56\") " pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:18.951456 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.951384    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct7mx\" (UniqueName: \"kubernetes.io/projected/f96ca98e-8ab7-4335-b49f-c97435d6eab7-kube-api-access-ct7mx\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cmlg4\" (UID: \"f96ca98e-8ab7-4335-b49f-c97435d6eab7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4"
Sep 07 08:59:18.951456 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:18.951440    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4spt\" (UniqueName: \"kubernetes.io/projected/d8454d84-c799-434e-b84f-aaee9b9a722c-kube-api-access-k4spt\") pod \"network-check-source-685456655f-6ds2f\" (UID: \"d8454d84-c799-434e-b84f-aaee9b9a722c\") " pod="openshift-network-diagnostics/network-check-source-685456655f-6ds2f"
Sep 07 08:59:19.015767 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.015739    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:19.051746 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.051683    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4spt\" (UniqueName: \"kubernetes.io/projected/d8454d84-c799-434e-b84f-aaee9b9a722c-kube-api-access-k4spt\") pod \"network-check-source-685456655f-6ds2f\" (UID: \"d8454d84-c799-434e-b84f-aaee9b9a722c\") " pod="openshift-network-diagnostics/network-check-source-685456655f-6ds2f"
Sep 07 08:59:19.051746 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.051726    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f96ca98e-8ab7-4335-b49f-c97435d6eab7-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cmlg4\" (UID: \"f96ca98e-8ab7-4335-b49f-c97435d6eab7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4"
Sep 07 08:59:19.051842 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.051766    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9kzsb\" (UID: \"a02c02cd-2f63-48de-995d-2c0887d00594\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"
Sep 07 08:59:19.051842 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.051788    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f96ca98e-8ab7-4335-b49f-c97435d6eab7-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cmlg4\" (UID: \"f96ca98e-8ab7-4335-b49f-c97435d6eab7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4"
Sep 07 08:59:19.051842 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.051804    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbwgj\" (UniqueName: \"kubernetes.io/projected/a02c02cd-2f63-48de-995d-2c0887d00594-kube-api-access-bbwgj\") pod \"cluster-samples-operator-7cdcfcf76c-9kzsb\" (UID: \"a02c02cd-2f63-48de-995d-2c0887d00594\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"
Sep 07 08:59:19.051842 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.051821    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgg4l\" (UniqueName: \"kubernetes.io/projected/b61be2b6-3472-4523-95d6-c964c440fe56-kube-api-access-bgg4l\") pod \"console-operator-85b454874b-hm7ht\" (UID: \"b61be2b6-3472-4523-95d6-c964c440fe56\") " pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:19.051842 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.051838    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/930228a4-40c6-4a1b-a2a1-599f1de15ec3-serving-cert\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.052065 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.051862    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-default-certificate\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.052065 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.051890    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f4d4c13f-bc7f-4f46-b32b-906e6136122b-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:19.052065 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052024    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-865kg\" (UniqueName: \"kubernetes.io/projected/c2c8a9a7-d1f6-48de-930c-c3250235f861-kube-api-access-865kg\") pod \"volume-data-source-validator-695c895bd6-9znqs\" (UID: \"c2c8a9a7-d1f6-48de-930c-c3250235f861\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-9znqs"
Sep 07 08:59:19.052065 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052057    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/930228a4-40c6-4a1b-a2a1-599f1de15ec3-snapshots\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.052265 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052084    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/930228a4-40c6-4a1b-a2a1-599f1de15ec3-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.052265 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052120    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b61be2b6-3472-4523-95d6-c964c440fe56-serving-cert\") pod \"console-operator-85b454874b-hm7ht\" (UID: \"b61be2b6-3472-4523-95d6-c964c440fe56\") " pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:19.052265 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052163    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2584a748-e024-4375-9635-2a9a21590230-nginx-conf\") pod \"networking-console-plugin-79977f9844-dbgqp\" (UID: \"2584a748-e024-4375-9635-2a9a21590230\") " pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp"
Sep 07 08:59:19.052265 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052209    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:19.052265 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052236    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/28768fce-452e-43c7-b924-4093e574e45a-serving-cert\") pod \"service-ca-operator-859f94c4b-wg754\" (UID: \"28768fce-452e-43c7-b924-4093e574e45a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754"
Sep 07 08:59:19.052265 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052260    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/930228a4-40c6-4a1b-a2a1-599f1de15ec3-tmp\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.052538 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052287    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmqsr\" (UniqueName: \"kubernetes.io/projected/930228a4-40c6-4a1b-a2a1-599f1de15ec3-kube-api-access-rmqsr\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.052538 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.052317    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 08:59:19.052538 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052349    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppm6s\" (UniqueName: \"kubernetes.io/projected/f4d4c13f-bc7f-4f46-b32b-906e6136122b-kube-api-access-ppm6s\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:19.052538 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052377    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b61be2b6-3472-4523-95d6-c964c440fe56-config\") pod \"console-operator-85b454874b-hm7ht\" (UID: \"b61be2b6-3472-4523-95d6-c964c440fe56\") " pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:19.052538 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.052404    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls podName:f4d4c13f-bc7f-4f46-b32b-906e6136122b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:19.552385816 +0000 UTC m=+42.516302017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-98mdq" (UID: "f4d4c13f-bc7f-4f46-b32b-906e6136122b") : secret "cluster-monitoring-operator-tls" not found
Sep 07 08:59:19.052538 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052407    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f96ca98e-8ab7-4335-b49f-c97435d6eab7-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cmlg4\" (UID: \"f96ca98e-8ab7-4335-b49f-c97435d6eab7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4"
Sep 07 08:59:19.052888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052548    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct7mx\" (UniqueName: \"kubernetes.io/projected/f96ca98e-8ab7-4335-b49f-c97435d6eab7-kube-api-access-ct7mx\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cmlg4\" (UID: \"f96ca98e-8ab7-4335-b49f-c97435d6eab7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4"
Sep 07 08:59:19.052888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052583    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.052888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052593    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f4d4c13f-bc7f-4f46-b32b-906e6136122b-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:19.052888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052637    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-dbgqp\" (UID: \"2584a748-e024-4375-9635-2a9a21590230\") " pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp"
Sep 07 08:59:19.052888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052670    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/28768fce-452e-43c7-b924-4093e574e45a-config\") pod \"service-ca-operator-859f94c4b-wg754\" (UID: \"28768fce-452e-43c7-b924-4093e574e45a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754"
Sep 07 08:59:19.052888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052689    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-stats-auth\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.052888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052725    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.052888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052887    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b61be2b6-3472-4523-95d6-c964c440fe56-trusted-ca\") pod \"console-operator-85b454874b-hm7ht\" (UID: \"b61be2b6-3472-4523-95d6-c964c440fe56\") " pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:19.053261 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052934    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/930228a4-40c6-4a1b-a2a1-599f1de15ec3-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.053261 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.052969    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqdlk\" (UniqueName: \"kubernetes.io/projected/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-kube-api-access-bqdlk\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.053261 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.053013    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6jgr\" (UniqueName: \"kubernetes.io/projected/28768fce-452e-43c7-b924-4093e574e45a-kube-api-access-m6jgr\") pod \"service-ca-operator-859f94c4b-wg754\" (UID: \"28768fce-452e-43c7-b924-4093e574e45a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754"
Sep 07 08:59:19.053261 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.053121    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b61be2b6-3472-4523-95d6-c964c440fe56-config\") pod \"console-operator-85b454874b-hm7ht\" (UID: \"b61be2b6-3472-4523-95d6-c964c440fe56\") " pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:19.054013 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.053990    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b61be2b6-3472-4523-95d6-c964c440fe56-trusted-ca\") pod \"console-operator-85b454874b-hm7ht\" (UID: \"b61be2b6-3472-4523-95d6-c964c440fe56\") " pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:19.055268 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.055250    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f96ca98e-8ab7-4335-b49f-c97435d6eab7-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cmlg4\" (UID: \"f96ca98e-8ab7-4335-b49f-c97435d6eab7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4"
Sep 07 08:59:19.055388 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.055372    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b61be2b6-3472-4523-95d6-c964c440fe56-serving-cert\") pod \"console-operator-85b454874b-hm7ht\" (UID: \"b61be2b6-3472-4523-95d6-c964c440fe56\") " pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:19.063596 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.063575    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgg4l\" (UniqueName: \"kubernetes.io/projected/b61be2b6-3472-4523-95d6-c964c440fe56-kube-api-access-bgg4l\") pod \"console-operator-85b454874b-hm7ht\" (UID: \"b61be2b6-3472-4523-95d6-c964c440fe56\") " pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:19.063895 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.063874    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppm6s\" (UniqueName: \"kubernetes.io/projected/f4d4c13f-bc7f-4f46-b32b-906e6136122b-kube-api-access-ppm6s\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:19.064130 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.064115    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct7mx\" (UniqueName: \"kubernetes.io/projected/f96ca98e-8ab7-4335-b49f-c97435d6eab7-kube-api-access-ct7mx\") pod \"kube-storage-version-migrator-operator-8d7dd4675-cmlg4\" (UID: \"f96ca98e-8ab7-4335-b49f-c97435d6eab7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4"
Sep 07 08:59:19.064174 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.064119    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4spt\" (UniqueName: \"kubernetes.io/projected/d8454d84-c799-434e-b84f-aaee9b9a722c-kube-api-access-k4spt\") pod \"network-check-source-685456655f-6ds2f\" (UID: \"d8454d84-c799-434e-b84f-aaee9b9a722c\") " pod="openshift-network-diagnostics/network-check-source-685456655f-6ds2f"
Sep 07 08:59:19.110130 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.110110    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-6ds2f"
Sep 07 08:59:19.115024 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.114997    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:19.121309 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.121293    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4"
Sep 07 08:59:19.153802 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.153782    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9kzsb\" (UID: \"a02c02cd-2f63-48de-995d-2c0887d00594\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"
Sep 07 08:59:19.153888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.153827    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbwgj\" (UniqueName: \"kubernetes.io/projected/a02c02cd-2f63-48de-995d-2c0887d00594-kube-api-access-bbwgj\") pod \"cluster-samples-operator-7cdcfcf76c-9kzsb\" (UID: \"a02c02cd-2f63-48de-995d-2c0887d00594\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"
Sep 07 08:59:19.153888 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.153861    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/930228a4-40c6-4a1b-a2a1-599f1de15ec3-serving-cert\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.153959 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.153886    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-default-certificate\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.153959 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.153906    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 08:59:19.154336 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.154314    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-865kg\" (UniqueName: \"kubernetes.io/projected/c2c8a9a7-d1f6-48de-930c-c3250235f861-kube-api-access-865kg\") pod \"volume-data-source-validator-695c895bd6-9znqs\" (UID: \"c2c8a9a7-d1f6-48de-930c-c3250235f861\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-9znqs"
Sep 07 08:59:19.154418 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.154360    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls podName:a02c02cd-2f63-48de-995d-2c0887d00594 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:19.654338061 +0000 UTC m=+42.618254271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-9kzsb" (UID: "a02c02cd-2f63-48de-995d-2c0887d00594") : secret "samples-operator-tls" not found
Sep 07 08:59:19.154418 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.154398    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/930228a4-40c6-4a1b-a2a1-599f1de15ec3-snapshots\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.154574 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.154429    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/930228a4-40c6-4a1b-a2a1-599f1de15ec3-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.154574 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.154491    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2584a748-e024-4375-9635-2a9a21590230-nginx-conf\") pod \"networking-console-plugin-79977f9844-dbgqp\" (UID: \"2584a748-e024-4375-9635-2a9a21590230\") " pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp"
Sep 07 08:59:19.154574 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.154544    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/28768fce-452e-43c7-b924-4093e574e45a-serving-cert\") pod \"service-ca-operator-859f94c4b-wg754\" (UID: \"28768fce-452e-43c7-b924-4093e574e45a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754"
Sep 07 08:59:19.154574 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.154574    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/930228a4-40c6-4a1b-a2a1-599f1de15ec3-tmp\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.154823 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.154610    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmqsr\" (UniqueName: \"kubernetes.io/projected/930228a4-40c6-4a1b-a2a1-599f1de15ec3-kube-api-access-rmqsr\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.154823 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.154696    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.154823 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.154766    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-dbgqp\" (UID: \"2584a748-e024-4375-9635-2a9a21590230\") " pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp"
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.154798    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/28768fce-452e-43c7-b924-4093e574e45a-config\") pod \"service-ca-operator-859f94c4b-wg754\" (UID: \"28768fce-452e-43c7-b924-4093e574e45a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754"
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.155444    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/28768fce-452e-43c7-b924-4093e574e45a-config\") pod \"service-ca-operator-859f94c4b-wg754\" (UID: \"28768fce-452e-43c7-b924-4093e574e45a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754"
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.156175    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2584a748-e024-4375-9635-2a9a21590230-nginx-conf\") pod \"networking-console-plugin-79977f9844-dbgqp\" (UID: \"2584a748-e024-4375-9635-2a9a21590230\") " pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp"
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.156420    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle podName:9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:19.656401407 +0000 UTC m=+42.620317622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle") pod "router-default-54ff77667d-jgqz5" (UID: "9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b") : configmap references non-existent config key: service-ca.crt
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.156635    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/930228a4-40c6-4a1b-a2a1-599f1de15ec3-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.156904    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-default-certificate\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.157032    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/930228a4-40c6-4a1b-a2a1-599f1de15ec3-snapshots\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.156497    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.157261    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert podName:2584a748-e024-4375-9635-2a9a21590230 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:19.657243329 +0000 UTC m=+42.621159535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-dbgqp" (UID: "2584a748-e024-4375-9635-2a9a21590230") : secret "networking-console-plugin-cert" not found
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.157285    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-stats-auth\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.157299    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/930228a4-40c6-4a1b-a2a1-599f1de15ec3-tmp\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.157316    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.157349    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/930228a4-40c6-4a1b-a2a1-599f1de15ec3-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.157398    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqdlk\" (UniqueName: \"kubernetes.io/projected/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-kube-api-access-bqdlk\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.157429    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 08:59:19.159640 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.157461    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6jgr\" (UniqueName: \"kubernetes.io/projected/28768fce-452e-43c7-b924-4093e574e45a-kube-api-access-m6jgr\") pod \"service-ca-operator-859f94c4b-wg754\" (UID: \"28768fce-452e-43c7-b924-4093e574e45a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754"
Sep 07 08:59:19.160423 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.157494    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs podName:9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:19.657469248 +0000 UTC m=+42.621385459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs") pod "router-default-54ff77667d-jgqz5" (UID: "9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b") : secret "router-metrics-certs-default" not found
Sep 07 08:59:19.160423 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.157623    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/930228a4-40c6-4a1b-a2a1-599f1de15ec3-serving-cert\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.160423 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.158020    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/930228a4-40c6-4a1b-a2a1-599f1de15ec3-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.160423 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.158527    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/28768fce-452e-43c7-b924-4093e574e45a-serving-cert\") pod \"service-ca-operator-859f94c4b-wg754\" (UID: \"28768fce-452e-43c7-b924-4093e574e45a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754"
Sep 07 08:59:19.162817 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.162791    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-865kg\" (UniqueName: \"kubernetes.io/projected/c2c8a9a7-d1f6-48de-930c-c3250235f861-kube-api-access-865kg\") pod \"volume-data-source-validator-695c895bd6-9znqs\" (UID: \"c2c8a9a7-d1f6-48de-930c-c3250235f861\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-9znqs"
Sep 07 08:59:19.162963 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.162915    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-stats-auth\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.166227 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.166207    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmqsr\" (UniqueName: \"kubernetes.io/projected/930228a4-40c6-4a1b-a2a1-599f1de15ec3-kube-api-access-rmqsr\") pod \"insights-operator-76bcd6ccf5-4ggpx\" (UID: \"930228a4-40c6-4a1b-a2a1-599f1de15ec3\") " pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.167285 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.167261    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6jgr\" (UniqueName: \"kubernetes.io/projected/28768fce-452e-43c7-b924-4093e574e45a-kube-api-access-m6jgr\") pod \"service-ca-operator-859f94c4b-wg754\" (UID: \"28768fce-452e-43c7-b924-4093e574e45a\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754"
Sep 07 08:59:19.167732 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.167698    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqdlk\" (UniqueName: \"kubernetes.io/projected/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-kube-api-access-bqdlk\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.168472 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.168458    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:19.174041 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.174020    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbwgj\" (UniqueName: \"kubernetes.io/projected/a02c02cd-2f63-48de-995d-2c0887d00594-kube-api-access-bbwgj\") pod \"cluster-samples-operator-7cdcfcf76c-9kzsb\" (UID: \"a02c02cd-2f63-48de-995d-2c0887d00594\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"
Sep 07 08:59:19.191023 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.190984    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx"
Sep 07 08:59:19.211109 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.211055    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754"
Sep 07 08:59:19.215442 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.215401    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:19.235991 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.235962    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-9znqs"
Sep 07 08:59:19.272597 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.272467    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:19.411477 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.411450    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-6ds2f"]
Sep 07 08:59:19.414551 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:19.414526    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8454d84_c799_434e_b84f_aaee9b9a722c.slice/crio-2b67e6c30c73429c00394001dd5160b032a0f2e3b9afc7d4998bd2b6f21d173a WatchSource:0}: Error finding container 2b67e6c30c73429c00394001dd5160b032a0f2e3b9afc7d4998bd2b6f21d173a: Status 404 returned error can't find the container with id 2b67e6c30c73429c00394001dd5160b032a0f2e3b9afc7d4998bd2b6f21d173a
Sep 07 08:59:19.438256 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.438211    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4"]
Sep 07 08:59:19.439965 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:19.439944    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf96ca98e_8ab7_4335_b49f_c97435d6eab7.slice/crio-53dab74284f31df6508c3de69b44cc1f8a3eac2b0ee8f87a3ccedebf64fbbd2b WatchSource:0}: Error finding container 53dab74284f31df6508c3de69b44cc1f8a3eac2b0ee8f87a3ccedebf64fbbd2b: Status 404 returned error can't find the container with id 53dab74284f31df6508c3de69b44cc1f8a3eac2b0ee8f87a3ccedebf64fbbd2b
Sep 07 08:59:19.488982 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.488965    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-hm7ht"]
Sep 07 08:59:19.491042 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:19.491024    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb61be2b6_3472_4523_95d6_c964c440fe56.slice/crio-e2e374ac47bd062e468482b2dcd9e97fa43e660b2a8729ac5f1b7b01b94e7713 WatchSource:0}: Error finding container e2e374ac47bd062e468482b2dcd9e97fa43e660b2a8729ac5f1b7b01b94e7713: Status 404 returned error can't find the container with id e2e374ac47bd062e468482b2dcd9e97fa43e660b2a8729ac5f1b7b01b94e7713
Sep 07 08:59:19.506788 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.506702    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-4ggpx"]
Sep 07 08:59:19.508809 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:19.508787    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod930228a4_40c6_4a1b_a2a1_599f1de15ec3.slice/crio-e0dc8673c27e971f7afe0dc0e418804ba57e18377287f9045cad3c49bc54bc13 WatchSource:0}: Error finding container e0dc8673c27e971f7afe0dc0e418804ba57e18377287f9045cad3c49bc54bc13: Status 404 returned error can't find the container with id e0dc8673c27e971f7afe0dc0e418804ba57e18377287f9045cad3c49bc54bc13
Sep 07 08:59:19.511233 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.511211    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754"]
Sep 07 08:59:19.513912 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:19.513885    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28768fce_452e_43c7_b924_4093e574e45a.slice/crio-1d2d2db94aa5e5cc66b26835801dadef212d56dda46d66fa30ce91dce81dc564 WatchSource:0}: Error finding container 1d2d2db94aa5e5cc66b26835801dadef212d56dda46d66fa30ce91dce81dc564: Status 404 returned error can't find the container with id 1d2d2db94aa5e5cc66b26835801dadef212d56dda46d66fa30ce91dce81dc564
Sep 07 08:59:19.529061 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.529037    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-9znqs"]
Sep 07 08:59:19.531496 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:19.531478    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2c8a9a7_d1f6_48de_930c_c3250235f861.slice/crio-dae97529823b90e420b6723e8812f16465404b19c64e622490e93ac09553b2a6 WatchSource:0}: Error finding container dae97529823b90e420b6723e8812f16465404b19c64e622490e93ac09553b2a6: Status 404 returned error can't find the container with id dae97529823b90e420b6723e8812f16465404b19c64e622490e93ac09553b2a6
Sep 07 08:59:19.560747 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.560692    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:19.560860 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.560846    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 08:59:19.560914 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.560905    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls podName:f4d4c13f-bc7f-4f46-b32b-906e6136122b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:20.560888978 +0000 UTC m=+43.524805179 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-98mdq" (UID: "f4d4c13f-bc7f-4f46-b32b-906e6136122b") : secret "cluster-monitoring-operator-tls" not found
Sep 07 08:59:19.662175 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.662149    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.662289 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.662190    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-dbgqp\" (UID: \"2584a748-e024-4375-9635-2a9a21590230\") " pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp"
Sep 07 08:59:19.662289 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.662216    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:19.662289 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.662259    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9kzsb\" (UID: \"a02c02cd-2f63-48de-995d-2c0887d00594\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"
Sep 07 08:59:19.662289 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.662285    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 08:59:19.662456 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.662302    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle podName:9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:20.662283624 +0000 UTC m=+43.626199842 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle") pod "router-default-54ff77667d-jgqz5" (UID: "9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b") : configmap references non-existent config key: service-ca.crt
Sep 07 08:59:19.662456 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.662332    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert podName:2584a748-e024-4375-9635-2a9a21590230 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:20.662322383 +0000 UTC m=+43.626238587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-dbgqp" (UID: "2584a748-e024-4375-9635-2a9a21590230") : secret "networking-console-plugin-cert" not found
Sep 07 08:59:19.662456 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.662333    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 08:59:19.662456 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.662348    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 08:59:19.662456 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.662371    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs podName:9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:20.662360682 +0000 UTC m=+43.626276883 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs") pod "router-default-54ff77667d-jgqz5" (UID: "9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b") : secret "router-metrics-certs-default" not found
Sep 07 08:59:19.662456 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:19.662394    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls podName:a02c02cd-2f63-48de-995d-2c0887d00594 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:20.66237787 +0000 UTC m=+43.626294085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-9kzsb" (UID: "a02c02cd-2f63-48de-995d-2c0887d00594") : secret "samples-operator-tls" not found
Sep 07 08:59:19.840706 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.840643    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754" event={"ID":"28768fce-452e-43c7-b924-4093e574e45a","Type":"ContainerStarted","Data":"1d2d2db94aa5e5cc66b26835801dadef212d56dda46d66fa30ce91dce81dc564"}
Sep 07 08:59:19.841856 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.841836    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-hm7ht" event={"ID":"b61be2b6-3472-4523-95d6-c964c440fe56","Type":"ContainerStarted","Data":"e2e374ac47bd062e468482b2dcd9e97fa43e660b2a8729ac5f1b7b01b94e7713"}
Sep 07 08:59:19.842920 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.842902    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4" event={"ID":"f96ca98e-8ab7-4335-b49f-c97435d6eab7","Type":"ContainerStarted","Data":"53dab74284f31df6508c3de69b44cc1f8a3eac2b0ee8f87a3ccedebf64fbbd2b"}
Sep 07 08:59:19.844072 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.844041    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-9znqs" event={"ID":"c2c8a9a7-d1f6-48de-930c-c3250235f861","Type":"ContainerStarted","Data":"dae97529823b90e420b6723e8812f16465404b19c64e622490e93ac09553b2a6"}
Sep 07 08:59:19.845144 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.845123    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-6ds2f" event={"ID":"d8454d84-c799-434e-b84f-aaee9b9a722c","Type":"ContainerStarted","Data":"2b67e6c30c73429c00394001dd5160b032a0f2e3b9afc7d4998bd2b6f21d173a"}
Sep 07 08:59:19.846098 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:19.846079    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx" event={"ID":"930228a4-40c6-4a1b-a2a1-599f1de15ec3","Type":"ContainerStarted","Data":"e0dc8673c27e971f7afe0dc0e418804ba57e18377287f9045cad3c49bc54bc13"}
Sep 07 08:59:20.166027 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:20.165971    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:20.166179 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:20.166085    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:20.166633 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:20.166611    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 08:59:20.166748 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:20.166697    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls podName:2e215d11-e0ca-493d-a261-35932fbea87c nodeName:}" failed. No retries permitted until 2026-09-07 08:59:28.16667551 +0000 UTC m=+51.130591723 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls") pod "dns-default-7g5t9" (UID: "2e215d11-e0ca-493d-a261-35932fbea87c") : secret "dns-default-metrics-tls" not found
Sep 07 08:59:20.167103 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:20.166400    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 08:59:20.167178 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:20.167108    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c9bc4456f-xpnlz: secret "image-registry-tls" not found
Sep 07 08:59:20.167178 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:20.167157    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls podName:5d8372a4-648d-4544-b928-faa61f9b936a nodeName:}" failed. No retries permitted until 2026-09-07 08:59:28.167142947 +0000 UTC m=+51.131059152 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls") pod "image-registry-c9bc4456f-xpnlz" (UID: "5d8372a4-648d-4544-b928-faa61f9b936a") : secret "image-registry-tls" not found
Sep 07 08:59:20.570986 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:20.570833    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:20.571148 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:20.571134    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 08:59:20.571210 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:20.571198    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls podName:f4d4c13f-bc7f-4f46-b32b-906e6136122b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:22.571179022 +0000 UTC m=+45.535095234 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-98mdq" (UID: "f4d4c13f-bc7f-4f46-b32b-906e6136122b") : secret "cluster-monitoring-operator-tls" not found
Sep 07 08:59:20.672218 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:20.672183    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:20.672371 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:20.672245    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-dbgqp\" (UID: \"2584a748-e024-4375-9635-2a9a21590230\") " pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp"
Sep 07 08:59:20.672371 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:20.672273    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:20.672371 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:20.672319    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9kzsb\" (UID: \"a02c02cd-2f63-48de-995d-2c0887d00594\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"
Sep 07 08:59:20.672543 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:20.672454    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 08:59:20.672543 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:20.672514    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls podName:a02c02cd-2f63-48de-995d-2c0887d00594 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:22.672495732 +0000 UTC m=+45.636411934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-9kzsb" (UID: "a02c02cd-2f63-48de-995d-2c0887d00594") : secret "samples-operator-tls" not found
Sep 07 08:59:20.673271 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:20.672940    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle podName:9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:22.67292317 +0000 UTC m=+45.636839370 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle") pod "router-default-54ff77667d-jgqz5" (UID: "9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b") : configmap references non-existent config key: service-ca.crt
Sep 07 08:59:20.673271 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:20.673023    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 08:59:20.673271 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:20.673059    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert podName:2584a748-e024-4375-9635-2a9a21590230 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:22.673048044 +0000 UTC m=+45.636964250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-dbgqp" (UID: "2584a748-e024-4375-9635-2a9a21590230") : secret "networking-console-plugin-cert" not found
Sep 07 08:59:20.673271 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:20.673115    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 08:59:20.673271 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:20.673146    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs podName:9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:22.673135257 +0000 UTC m=+45.637051459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs") pod "router-default-54ff77667d-jgqz5" (UID: "9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b") : secret "router-metrics-certs-default" not found
Sep 07 08:59:22.590191 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:22.590150    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:22.590654 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:22.590313    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 08:59:22.590654 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:22.590396    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls podName:f4d4c13f-bc7f-4f46-b32b-906e6136122b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:26.590373181 +0000 UTC m=+49.554289382 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-98mdq" (UID: "f4d4c13f-bc7f-4f46-b32b-906e6136122b") : secret "cluster-monitoring-operator-tls" not found
Sep 07 08:59:22.691436 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:22.691401    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9kzsb\" (UID: \"a02c02cd-2f63-48de-995d-2c0887d00594\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"
Sep 07 08:59:22.691612 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:22.691539    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:22.691612 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:22.691560    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 08:59:22.691741 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:22.691619    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls podName:a02c02cd-2f63-48de-995d-2c0887d00594 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:26.691605207 +0000 UTC m=+49.655521406 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-9kzsb" (UID: "a02c02cd-2f63-48de-995d-2c0887d00594") : secret "samples-operator-tls" not found
Sep 07 08:59:22.691741 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:22.691565    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-dbgqp\" (UID: \"2584a748-e024-4375-9635-2a9a21590230\") " pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp"
Sep 07 08:59:22.691741 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:22.691658    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:22.691741 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:22.691630    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 08:59:22.691741 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:22.691706    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle podName:9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:26.691687726 +0000 UTC m=+49.655603929 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle") pod "router-default-54ff77667d-jgqz5" (UID: "9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b") : configmap references non-existent config key: service-ca.crt
Sep 07 08:59:22.692002 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:22.691759    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert podName:2584a748-e024-4375-9635-2a9a21590230 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:26.691744427 +0000 UTC m=+49.655660649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-dbgqp" (UID: "2584a748-e024-4375-9635-2a9a21590230") : secret "networking-console-plugin-cert" not found
Sep 07 08:59:22.692002 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:22.691767    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 08:59:22.692002 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:22.691803    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs podName:9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:26.691792026 +0000 UTC m=+49.655708246 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs") pod "router-default-54ff77667d-jgqz5" (UID: "9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b") : secret "router-metrics-certs-default" not found
Sep 07 08:59:24.210345 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:24.210297    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:24.309576 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:24.309542    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:24.791902 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:24.791866    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:24.846078 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:24.846033    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:24.898540 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:24.898495    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:25.216684 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:25.216650    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:25.550643 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:25.550558    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:25.606898 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:25.606867    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:26.350618 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.350588    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:26.626011 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.625969    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:26.626199 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:26.626142    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 08:59:26.626254 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:26.626227    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls podName:f4d4c13f-bc7f-4f46-b32b-906e6136122b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:34.626205749 +0000 UTC m=+57.590121954 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-98mdq" (UID: "f4d4c13f-bc7f-4f46-b32b-906e6136122b") : secret "cluster-monitoring-operator-tls" not found
Sep 07 08:59:26.727070 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.726987    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:26.727070 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.727045    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-dbgqp\" (UID: \"2584a748-e024-4375-9635-2a9a21590230\") " pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp"
Sep 07 08:59:26.727244 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:26.727146    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 08:59:26.727244 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:26.727179    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle podName:9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:34.727156057 +0000 UTC m=+57.691072270 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle") pod "router-default-54ff77667d-jgqz5" (UID: "9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b") : configmap references non-existent config key: service-ca.crt
Sep 07 08:59:26.727244 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:26.727206    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert podName:2584a748-e024-4375-9635-2a9a21590230 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:34.727196836 +0000 UTC m=+57.691113040 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-dbgqp" (UID: "2584a748-e024-4375-9635-2a9a21590230") : secret "networking-console-plugin-cert" not found
Sep 07 08:59:26.727244 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.727228    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:26.727439 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.727281    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9kzsb\" (UID: \"a02c02cd-2f63-48de-995d-2c0887d00594\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"
Sep 07 08:59:26.727439 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:26.727363    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 08:59:26.727439 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:26.727379    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 08:59:26.727439 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:26.727417    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs podName:9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:34.727400401 +0000 UTC m=+57.691316614 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs") pod "router-default-54ff77667d-jgqz5" (UID: "9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b") : secret "router-metrics-certs-default" not found
Sep 07 08:59:26.727439 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:26.727439    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls podName:a02c02cd-2f63-48de-995d-2c0887d00594 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:34.727423789 +0000 UTC m=+57.691340007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-9kzsb" (UID: "a02c02cd-2f63-48de-995d-2c0887d00594") : secret "samples-operator-tls" not found
Sep 07 08:59:26.776580 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.776554    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:26.826525 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.826500    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:26.871497 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.871476    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754" event={"ID":"28768fce-452e-43c7-b924-4093e574e45a","Type":"ContainerStarted","Data":"fb92cfd974e0b26d5f53e5ef37621f4171fa5ca110485373a0f9df779146da39"}
Sep 07 08:59:26.873563 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.873545    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/0.log"
Sep 07 08:59:26.873658 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.873584    2574 generic.go:356] "Generic (PLEG): container finished" podID="b61be2b6-3472-4523-95d6-c964c440fe56" containerID="f96aabdfa19528db272a9302d5509039c0a51deaeb437bd8981a4b5369c09d95" exitCode=255
Sep 07 08:59:26.873739 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.873657    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-hm7ht" event={"ID":"b61be2b6-3472-4523-95d6-c964c440fe56","Type":"ContainerDied","Data":"f96aabdfa19528db272a9302d5509039c0a51deaeb437bd8981a4b5369c09d95"}
Sep 07 08:59:26.873879 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.873863    2574 scope.go:117] "RemoveContainer" containerID="f96aabdfa19528db272a9302d5509039c0a51deaeb437bd8981a4b5369c09d95"
Sep 07 08:59:26.875470 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.875418    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4" event={"ID":"f96ca98e-8ab7-4335-b49f-c97435d6eab7","Type":"ContainerStarted","Data":"c1d1ababcda362681004aeda175aca644adf5c14ad525be653f68afa8f77c04e"}
Sep 07 08:59:26.877164 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.877128    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-9znqs" event={"ID":"c2c8a9a7-d1f6-48de-930c-c3250235f861","Type":"ContainerStarted","Data":"33638f04fb43456126a6745441b9d41a3dd665327ba7da0c6fe99cf826efb73c"}
Sep 07 08:59:26.879037 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.879017    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-6ds2f" event={"ID":"d8454d84-c799-434e-b84f-aaee9b9a722c","Type":"ContainerStarted","Data":"45c60aead3a7b31bcaa93ba2343f1230b65864dea6e042bbb0418a7898474b37"}
Sep 07 08:59:26.881205 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.881185    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx" event={"ID":"930228a4-40c6-4a1b-a2a1-599f1de15ec3","Type":"ContainerStarted","Data":"5b17449038c84e9614420aebb077bb800c90a73cf43e0da4a1d2d36ad84a1f69"}
Sep 07 08:59:26.887268 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.887223    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754" podStartSLOduration=2.198672338 podStartE2EDuration="8.887206257s" totalImagesPullingTime="6.688533919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:26.885590716 +0000 UTC m=+49.849506937" watchObservedRunningTime="2026-09-07 08:59:26.887206257 +0000 UTC m=+49.851122480"
Sep 07 08:59:26.904605 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.904205    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4" podStartSLOduration=2.141762021 podStartE2EDuration="8.904189512s" totalImagesPullingTime="6.762427491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:26.902930572 +0000 UTC m=+49.866846794" watchObservedRunningTime="2026-09-07 08:59:26.904189512 +0000 UTC m=+49.868105735"
Sep 07 08:59:26.917429 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.917366    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx" podStartSLOduration=2.223863759 podStartE2EDuration="8.917351125s" totalImagesPullingTime="6.693487366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:26.916990033 +0000 UTC m=+49.880906278" watchObservedRunningTime="2026-09-07 08:59:26.917351125 +0000 UTC m=+49.881267343"
Sep 07 08:59:26.954233 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.952953    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-9znqs" podStartSLOduration=2.282561255 podStartE2EDuration="8.952935339s" totalImagesPullingTime="6.670374084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:26.93698973 +0000 UTC m=+49.900905952" watchObservedRunningTime="2026-09-07 08:59:26.952935339 +0000 UTC m=+49.916851564"
Sep 07 08:59:26.970074 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:26.970030    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-6ds2f" podStartSLOduration=2.173091572 podStartE2EDuration="8.970013312s" totalImagesPullingTime="6.79692174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:26.969667133 +0000 UTC m=+49.933583387" watchObservedRunningTime="2026-09-07 08:59:26.970013312 +0000 UTC m=+49.933929534"
Sep 07 08:59:27.013166 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.013132    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:27.074654 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.074631    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:27.248163 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.248133    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pn9pp"]
Sep 07 08:59:27.260691 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.260661    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pn9pp"]
Sep 07 08:59:27.260837 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.260815    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.262933 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.262911    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 07 08:59:27.263413 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.263186    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vhtvf\""
Sep 07 08:59:27.263413 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.263301    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 07 08:59:27.269829 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.269806    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:27.325037 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.325016    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:27.335075 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.335054    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba613bd6-8f5b-4edc-8a06-b28348288069-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.335205 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.335186    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h52r7\" (UniqueName: \"kubernetes.io/projected/ba613bd6-8f5b-4edc-8a06-b28348288069-kube-api-access-h52r7\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.335261 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.335237    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba613bd6-8f5b-4edc-8a06-b28348288069-data-volume\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.335316 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.335267    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ba613bd6-8f5b-4edc-8a06-b28348288069-crio-socket\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.335363 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.335319    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.436727 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.436687    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h52r7\" (UniqueName: \"kubernetes.io/projected/ba613bd6-8f5b-4edc-8a06-b28348288069-kube-api-access-h52r7\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.437076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.436741    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba613bd6-8f5b-4edc-8a06-b28348288069-data-volume\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.437076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.436776    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ba613bd6-8f5b-4edc-8a06-b28348288069-crio-socket\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.437076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.436863    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.437076 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.436933    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba613bd6-8f5b-4edc-8a06-b28348288069-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.437076 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:27.437023    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 08:59:27.437076 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:27.437073    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 08:59:27.437357 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:27.437086    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ba613bd6-8f5b-4edc-8a06-b28348288069-kube-rbac-proxy-cm podName:ba613bd6-8f5b-4edc-8a06-b28348288069 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:27.937068185 +0000 UTC m=+50.900984401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ba613bd6-8f5b-4edc-8a06-b28348288069-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pn9pp" (UID: "ba613bd6-8f5b-4edc-8a06-b28348288069") : configmap "kube-rbac-proxy" not found
Sep 07 08:59:27.437357 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:27.437111    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls podName:ba613bd6-8f5b-4edc-8a06-b28348288069 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:27.937098389 +0000 UTC m=+50.901014589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pn9pp" (UID: "ba613bd6-8f5b-4edc-8a06-b28348288069") : secret "insights-runtime-extractor-tls" not found
Sep 07 08:59:27.437357 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.437071    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ba613bd6-8f5b-4edc-8a06-b28348288069-crio-socket\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.437357 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.437155    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba613bd6-8f5b-4edc-8a06-b28348288069-data-volume\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.444794 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.444766    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h52r7\" (UniqueName: \"kubernetes.io/projected/ba613bd6-8f5b-4edc-8a06-b28348288069-kube-api-access-h52r7\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.717189 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.717159    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-2hqr9"]
Sep 07 08:59:27.727901 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.727870    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:27.741403 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.741379    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-2hqr9"]
Sep 07 08:59:27.741539 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.741487    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-2hqr9"
Sep 07 08:59:27.743560 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.743540    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 07 08:59:27.743755 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.743574    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 07 08:59:27.743870 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.743849    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-wkzst\""
Sep 07 08:59:27.820911 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.820885    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:27.841177 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.841151    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km6c9\" (UniqueName: \"kubernetes.io/projected/28974197-205e-48ac-abbb-b57111693793-kube-api-access-km6c9\") pod \"migrator-fdbb8fdff-2hqr9\" (UID: \"28974197-205e-48ac-abbb-b57111693793\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-2hqr9"
Sep 07 08:59:27.886136 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.886120    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/1.log"
Sep 07 08:59:27.886577 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.886563    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/0.log"
Sep 07 08:59:27.886667 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.886594    2574 generic.go:356] "Generic (PLEG): container finished" podID="b61be2b6-3472-4523-95d6-c964c440fe56" containerID="e17f10ac5d8c6dbcea874c71560fae942041c08a96e6357100288673e1dd2775" exitCode=255
Sep 07 08:59:27.886737 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.886690    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-hm7ht" event={"ID":"b61be2b6-3472-4523-95d6-c964c440fe56","Type":"ContainerDied","Data":"e17f10ac5d8c6dbcea874c71560fae942041c08a96e6357100288673e1dd2775"}
Sep 07 08:59:27.886788 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.886761    2574 scope.go:117] "RemoveContainer" containerID="f96aabdfa19528db272a9302d5509039c0a51deaeb437bd8981a4b5369c09d95"
Sep 07 08:59:27.887105 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.886939    2574 scope.go:117] "RemoveContainer" containerID="e17f10ac5d8c6dbcea874c71560fae942041c08a96e6357100288673e1dd2775"
Sep 07 08:59:27.887198 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:27.887107    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-hm7ht_openshift-console-operator(b61be2b6-3472-4523-95d6-c964c440fe56)\"" pod="openshift-console-operator/console-operator-85b454874b-hm7ht" podUID="b61be2b6-3472-4523-95d6-c964c440fe56"
Sep 07 08:59:27.905652 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.905629    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:27.942162 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.942138    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.942255 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.942206    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km6c9\" (UniqueName: \"kubernetes.io/projected/28974197-205e-48ac-abbb-b57111693793-kube-api-access-km6c9\") pod \"migrator-fdbb8fdff-2hqr9\" (UID: \"28974197-205e-48ac-abbb-b57111693793\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-2hqr9"
Sep 07 08:59:27.942319 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:27.942276    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 08:59:27.942378 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.942341    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba613bd6-8f5b-4edc-8a06-b28348288069-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:27.942514 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:27.942427    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 08:59:27.942514 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:27.942483    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls podName:ba613bd6-8f5b-4edc-8a06-b28348288069 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:28.942455286 +0000 UTC m=+51.906371492 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pn9pp" (UID: "ba613bd6-8f5b-4edc-8a06-b28348288069") : secret "insights-runtime-extractor-tls" not found
Sep 07 08:59:27.943098 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:27.942814    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ba613bd6-8f5b-4edc-8a06-b28348288069-kube-rbac-proxy-cm podName:ba613bd6-8f5b-4edc-8a06-b28348288069 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:28.942799445 +0000 UTC m=+51.906715649 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ba613bd6-8f5b-4edc-8a06-b28348288069-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pn9pp" (UID: "ba613bd6-8f5b-4edc-8a06-b28348288069") : configmap "kube-rbac-proxy" not found
Sep 07 08:59:27.952009 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:27.951990    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km6c9\" (UniqueName: \"kubernetes.io/projected/28974197-205e-48ac-abbb-b57111693793-kube-api-access-km6c9\") pod \"migrator-fdbb8fdff-2hqr9\" (UID: \"28974197-205e-48ac-abbb-b57111693793\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-2hqr9"
Sep 07 08:59:28.050555 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.050503    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-2hqr9"
Sep 07 08:59:28.159209 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.159176    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:28.245181 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.245154    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:28.245314 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.245254    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:28.245392 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:28.245338    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 08:59:28.245441 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:28.245392    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 08:59:28.245441 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:28.245405    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c9bc4456f-xpnlz: secret "image-registry-tls" not found
Sep 07 08:59:28.245441 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:28.245415    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls podName:2e215d11-e0ca-493d-a261-35932fbea87c nodeName:}" failed. No retries permitted until 2026-09-07 08:59:44.245393896 +0000 UTC m=+67.209310105 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls") pod "dns-default-7g5t9" (UID: "2e215d11-e0ca-493d-a261-35932fbea87c") : secret "dns-default-metrics-tls" not found
Sep 07 08:59:28.245576 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:28.245447    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls podName:5d8372a4-648d-4544-b928-faa61f9b936a nodeName:}" failed. No retries permitted until 2026-09-07 08:59:44.245434037 +0000 UTC m=+67.209350236 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls") pod "image-registry-c9bc4456f-xpnlz" (UID: "5d8372a4-648d-4544-b928-faa61f9b936a") : secret "image-registry-tls" not found
Sep 07 08:59:28.343372 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.343346    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-2hqr9"]
Sep 07 08:59:28.345738 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:28.345683    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28974197_205e_48ac_abbb_b57111693793.slice/crio-db0732fb127756457818ff2874176503028f3aaa8e413179494e65402b1fdfa7 WatchSource:0}: Error finding container db0732fb127756457818ff2874176503028f3aaa8e413179494e65402b1fdfa7: Status 404 returned error can't find the container with id db0732fb127756457818ff2874176503028f3aaa8e413179494e65402b1fdfa7
Sep 07 08:59:28.399218 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.399194    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:28.499528 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.499502    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:28.640554 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.640502    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rpdbr_40ddcfef-57f6-46fd-8d22-d4d0b5da9647/dns-node-resolver/0.log"
Sep 07 08:59:28.750466 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.750435    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:28.860496 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.860464    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:28.893840 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.893776    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/1.log"
Sep 07 08:59:28.894115 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.894096    2574 scope.go:117] "RemoveContainer" containerID="e17f10ac5d8c6dbcea874c71560fae942041c08a96e6357100288673e1dd2775"
Sep 07 08:59:28.894291 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:28.894271    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-hm7ht_openshift-console-operator(b61be2b6-3472-4523-95d6-c964c440fe56)\"" pod="openshift-console-operator/console-operator-85b454874b-hm7ht" podUID="b61be2b6-3472-4523-95d6-c964c440fe56"
Sep 07 08:59:28.895256 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.895228    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-2hqr9" event={"ID":"28974197-205e-48ac-abbb-b57111693793","Type":"ContainerStarted","Data":"db0732fb127756457818ff2874176503028f3aaa8e413179494e65402b1fdfa7"}
Sep 07 08:59:28.917207 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.917179    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:28.951269 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.951239    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:28.951427 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.951315    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba613bd6-8f5b-4edc-8a06-b28348288069-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:28.951491 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:28.951457    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 08:59:28.951544 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:28.951525    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls podName:ba613bd6-8f5b-4edc-8a06-b28348288069 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:30.951504001 +0000 UTC m=+53.915420216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pn9pp" (UID: "ba613bd6-8f5b-4edc-8a06-b28348288069") : secret "insights-runtime-extractor-tls" not found
Sep 07 08:59:28.951882 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:28.951863    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba613bd6-8f5b-4edc-8a06-b28348288069-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:29.137893 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:29.137851    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:29.169094 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:29.169071    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:29.169227 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:29.169123    2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:29.240932 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:29.240893    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:29.241289 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:29.241275    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4s9pv_fa333c43-6b4b-4023-8665-bd82f6943f20/node-ca/0.log"
Sep 07 08:59:29.712143 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:29.712112    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:29.811010 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:29.810975    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:29.898039 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:29.898019    2574 scope.go:117] "RemoveContainer" containerID="e17f10ac5d8c6dbcea874c71560fae942041c08a96e6357100288673e1dd2775"
Sep 07 08:59:29.898213 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:29.898193    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-hm7ht_openshift-console-operator(b61be2b6-3472-4523-95d6-c964c440fe56)\"" pod="openshift-console-operator/console-operator-85b454874b-hm7ht" podUID="b61be2b6-3472-4523-95d6-c964c440fe56"
Sep 07 08:59:30.040940 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.040918    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hzt78_c82b73b1-7d84-4d8f-be08-c7cf655fe39d/serve-healthcheck-canary/0.log"
Sep 07 08:59:30.731751 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.731663    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-qxphb"]
Sep 07 08:59:30.750079 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.750057    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-qxphb"]
Sep 07 08:59:30.750221 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.750159    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-qxphb"
Sep 07 08:59:30.752088 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.752069    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 07 08:59:30.752286 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.752271    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 07 08:59:30.752343 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.752272    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9rb42\""
Sep 07 08:59:30.752558 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.752543    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 07 08:59:30.752642 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.752546    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 07 08:59:30.773406 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.773381    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:30.822440 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.822420    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:30.867789 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.867763    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsxkn\" (UniqueName: \"kubernetes.io/projected/fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1-kube-api-access-vsxkn\") pod \"service-ca-75d99c484-qxphb\" (UID: \"fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1\") " pod="openshift-service-ca/service-ca-75d99c484-qxphb"
Sep 07 08:59:30.867873 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.867816    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1-signing-key\") pod \"service-ca-75d99c484-qxphb\" (UID: \"fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1\") " pod="openshift-service-ca/service-ca-75d99c484-qxphb"
Sep 07 08:59:30.867873 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.867835    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1-signing-cabundle\") pod \"service-ca-75d99c484-qxphb\" (UID: \"fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1\") " pod="openshift-service-ca/service-ca-75d99c484-qxphb"
Sep 07 08:59:30.904027 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.903995    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-2hqr9" event={"ID":"28974197-205e-48ac-abbb-b57111693793","Type":"ContainerStarted","Data":"c8559f21c473403786f19ea98e1cd660e0826b6ed2084815356b3c663c227595"}
Sep 07 08:59:30.904129 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.904090    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-2hqr9" event={"ID":"28974197-205e-48ac-abbb-b57111693793","Type":"ContainerStarted","Data":"0c37882ec80e915c28952c2ab9ce53934c614feba3f387fb2cf5cb2bf348325a"}
Sep 07 08:59:30.934600 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.934560    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-2hqr9" podStartSLOduration=2.396875928 podStartE2EDuration="3.934546665s" totalImagesPullingTime="1.537670737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:30.933372523 +0000 UTC m=+53.897288745" watchObservedRunningTime="2026-09-07 08:59:30.934546665 +0000 UTC m=+53.898462885"
Sep 07 08:59:30.968768 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.968746    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1-signing-key\") pod \"service-ca-75d99c484-qxphb\" (UID: \"fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1\") " pod="openshift-service-ca/service-ca-75d99c484-qxphb"
Sep 07 08:59:30.968862 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.968772    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1-signing-cabundle\") pod \"service-ca-75d99c484-qxphb\" (UID: \"fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1\") " pod="openshift-service-ca/service-ca-75d99c484-qxphb"
Sep 07 08:59:30.968862 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.968801    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:30.968951 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:30.968938    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 08:59:30.969008 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.968947    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsxkn\" (UniqueName: \"kubernetes.io/projected/fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1-kube-api-access-vsxkn\") pod \"service-ca-75d99c484-qxphb\" (UID: \"fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1\") " pod="openshift-service-ca/service-ca-75d99c484-qxphb"
Sep 07 08:59:30.969008 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:30.968997    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls podName:ba613bd6-8f5b-4edc-8a06-b28348288069 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:34.968980556 +0000 UTC m=+57.932896758 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pn9pp" (UID: "ba613bd6-8f5b-4edc-8a06-b28348288069") : secret "insights-runtime-extractor-tls" not found
Sep 07 08:59:30.969426 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.969403    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1-signing-cabundle\") pod \"service-ca-75d99c484-qxphb\" (UID: \"fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1\") " pod="openshift-service-ca/service-ca-75d99c484-qxphb"
Sep 07 08:59:30.971652 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.971633    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1-signing-key\") pod \"service-ca-75d99c484-qxphb\" (UID: \"fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1\") " pod="openshift-service-ca/service-ca-75d99c484-qxphb"
Sep 07 08:59:30.981069 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:30.981050    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsxkn\" (UniqueName: \"kubernetes.io/projected/fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1-kube-api-access-vsxkn\") pod \"service-ca-75d99c484-qxphb\" (UID: \"fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1\") " pod="openshift-service-ca/service-ca-75d99c484-qxphb"
Sep 07 08:59:31.009360 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:31.009299    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:31.059463 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:31.059439    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-qxphb"
Sep 07 08:59:31.061497 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:31.061477    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:31.342601 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:31.342522    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-qxphb"]
Sep 07 08:59:31.348696 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:31.348671    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd7c5a74_29c6_4f70_a9e6_cb0f0db59cb1.slice/crio-a1ac2c23cb0c4985e76d06b86e150ad6f43ec02c63b7007cc39d9ee5de131ee4 WatchSource:0}: Error finding container a1ac2c23cb0c4985e76d06b86e150ad6f43ec02c63b7007cc39d9ee5de131ee4: Status 404 returned error can't find the container with id a1ac2c23cb0c4985e76d06b86e150ad6f43ec02c63b7007cc39d9ee5de131ee4
Sep 07 08:59:31.698213 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:31.698171    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:31.909507 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:31.909463    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-qxphb" event={"ID":"fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1","Type":"ContainerStarted","Data":"5cdf92baa10ffe57bdb3b4879ffca0e28f2421009d468985c82dc696f546dc50"}
Sep 07 08:59:31.909920 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:31.909527    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-qxphb" event={"ID":"fd7c5a74-29c6-4f70-a9e6-cb0f0db59cb1","Type":"ContainerStarted","Data":"a1ac2c23cb0c4985e76d06b86e150ad6f43ec02c63b7007cc39d9ee5de131ee4"}
Sep 07 08:59:31.924878 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:31.924834    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-qxphb" podStartSLOduration=1.924819772 podStartE2EDuration="1.924819772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:31.923507262 +0000 UTC m=+54.887423480" watchObservedRunningTime="2026-09-07 08:59:31.924819772 +0000 UTC m=+54.888736041"
Sep 07 08:59:32.337605 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:32.337567    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:32.547742 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:32.547679    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:32.647657 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:32.647575    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:32.757056 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:32.757016    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:32.862636 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:32.862602    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:33.432121 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:33.432091    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:34.026865 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:34.026832    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:34.177621 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:34.177588    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:34.278674 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:34.278577    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:34.425038 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:34.425006    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:34.704408 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:34.704376    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:34.704899 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:34.704555    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 08:59:34.704899 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:34.704640    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls podName:f4d4c13f-bc7f-4f46-b32b-906e6136122b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:50.704618963 +0000 UTC m=+73.668535174 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-98mdq" (UID: "f4d4c13f-bc7f-4f46-b32b-906e6136122b") : secret "cluster-monitoring-operator-tls" not found
Sep 07 08:59:34.805665 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:34.805634    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9kzsb\" (UID: \"a02c02cd-2f63-48de-995d-2c0887d00594\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"
Sep 07 08:59:34.805849 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:34.805834    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:34.805895 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:34.805875    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-dbgqp\" (UID: \"2584a748-e024-4375-9635-2a9a21590230\") " pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp"
Sep 07 08:59:34.805983 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:34.805902    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:34.806026 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:34.805980    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 08:59:34.806026 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:34.805996    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 08:59:34.806026 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:34.806000    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle podName:9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:50.805978234 +0000 UTC m=+73.769894439 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle") pod "router-default-54ff77667d-jgqz5" (UID: "9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b") : configmap references non-existent config key: service-ca.crt
Sep 07 08:59:34.806126 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:34.806049    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert podName:2584a748-e024-4375-9635-2a9a21590230 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:50.806033582 +0000 UTC m=+73.769949784 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-dbgqp" (UID: "2584a748-e024-4375-9635-2a9a21590230") : secret "networking-console-plugin-cert" not found
Sep 07 08:59:34.806126 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:34.806065    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs podName:9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b nodeName:}" failed. No retries permitted until 2026-09-07 08:59:50.806055285 +0000 UTC m=+73.769971487 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs") pod "router-default-54ff77667d-jgqz5" (UID: "9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b") : secret "router-metrics-certs-default" not found
Sep 07 08:59:34.807936 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:34.807917    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a02c02cd-2f63-48de-995d-2c0887d00594-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-9kzsb\" (UID: \"a02c02cd-2f63-48de-995d-2c0887d00594\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"
Sep 07 08:59:34.825057 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:34.825040    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"
Sep 07 08:59:34.827696 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:34.827557    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:35.008167 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:35.008095    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:35.008300 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:35.008195    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 08:59:35.008300 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:35.008261    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls podName:ba613bd6-8f5b-4edc-8a06-b28348288069 nodeName:}" failed. No retries permitted until 2026-09-07 08:59:43.008243694 +0000 UTC m=+65.972159897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pn9pp" (UID: "ba613bd6-8f5b-4edc-8a06-b28348288069") : secret "insights-runtime-extractor-tls" not found
Sep 07 08:59:35.094211 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:35.094189    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb"]
Sep 07 08:59:35.420753 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:35.420705    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:35.834699 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:35.834625    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:35.924548 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:35.924504    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb" event={"ID":"a02c02cd-2f63-48de-995d-2c0887d00594","Type":"ContainerStarted","Data":"a039346669d27eb9e9962f09158d6338cb4749763e375e387d8eda7bbb958284"}
Sep 07 08:59:36.026521 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:36.026488    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:36.077646 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:36.077610    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:36.407917 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:36.407874    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:36.461177 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:36.461141    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:36.929443 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:36.929410    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb" event={"ID":"a02c02cd-2f63-48de-995d-2c0887d00594","Type":"ContainerStarted","Data":"cd8e6facfa3903d98e41beb8ae3ed4016e34910310a0b460394e545a87fb151c"}
Sep 07 08:59:37.086346 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:37.086271    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:37.140297 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:37.140268    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:37.426578 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:37.426548    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:37.623958 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:37.623927    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:37.810038 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:37.809982    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zdlqw"
Sep 07 08:59:37.933957 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:37.933930    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb" event={"ID":"a02c02cd-2f63-48de-995d-2c0887d00594","Type":"ContainerStarted","Data":"1a5b386f99ec89c668aefe0e059964031b538a85734dedd6e84f3b0dc277be1a"}
Sep 07 08:59:37.950581 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:37.950523    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-9kzsb" podStartSLOduration=18.294424008 podStartE2EDuration="19.950497126s" totalImagesPullingTime="1.656073118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:37.949121865 +0000 UTC m=+60.913038107" watchObservedRunningTime="2026-09-07 08:59:37.950497126 +0000 UTC m=+60.914413348"
Sep 07 08:59:38.220502 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:38.220476    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:38.453331 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:38.453303    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:38.630059 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:38.629982    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:38.681763 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:38.681737    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:38.836092 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:38.836061    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:39.029650 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:39.029614    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:40.022727 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:40.022670    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:40.223604 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:40.223556    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:40.400827 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:40.400742    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:40.456348 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:40.456319    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:40.536313 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:40.536289    2574 scope.go:117] "RemoveContainer" containerID="e17f10ac5d8c6dbcea874c71560fae942041c08a96e6357100288673e1dd2775"
Sep 07 08:59:40.621547 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:40.621514    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:40.828540 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:40.828497    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:40.945014 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:40.944992    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 08:59:40.945425 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:40.945411    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/1.log"
Sep 07 08:59:40.945470 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:40.945442    2574 generic.go:356] "Generic (PLEG): container finished" podID="b61be2b6-3472-4523-95d6-c964c440fe56" containerID="21485c42e4e99342694069859dbd909bb3538f06304b53033054ee399e02da50" exitCode=255
Sep 07 08:59:40.945516 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:40.945483    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-hm7ht" event={"ID":"b61be2b6-3472-4523-95d6-c964c440fe56","Type":"ContainerDied","Data":"21485c42e4e99342694069859dbd909bb3538f06304b53033054ee399e02da50"}
Sep 07 08:59:40.945554 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:40.945537    2574 scope.go:117] "RemoveContainer" containerID="e17f10ac5d8c6dbcea874c71560fae942041c08a96e6357100288673e1dd2775"
Sep 07 08:59:40.945829 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:40.945815    2574 scope.go:117] "RemoveContainer" containerID="21485c42e4e99342694069859dbd909bb3538f06304b53033054ee399e02da50"
Sep 07 08:59:40.946014 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:40.945996    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-hm7ht_openshift-console-operator(b61be2b6-3472-4523-95d6-c964c440fe56)\"" pod="openshift-console-operator/console-operator-85b454874b-hm7ht" podUID="b61be2b6-3472-4523-95d6-c964c440fe56"
Sep 07 08:59:41.022668 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:41.022625    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:41.613032 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:41.613004    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:41.664463 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:41.664442    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:41.952963 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:41.952940    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 08:59:42.227378 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:42.227309    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:42.328242 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:42.328209    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:42.433550 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:42.433521    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:42.623724 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:42.623635    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:42.825170 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:42.825142    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:43.026341 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.026295    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:43.083482 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.083452    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:43.085895 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.085877    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba613bd6-8f5b-4edc-8a06-b28348288069-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pn9pp\" (UID: \"ba613bd6-8f5b-4edc-8a06-b28348288069\") " pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:43.174936 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.174915    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vhtvf\""
Sep 07 08:59:43.183466 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.183448    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pn9pp"
Sep 07 08:59:43.285362 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.285299    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs\") pod \"network-metrics-daemon-c594r\" (UID: \"2aee0981-afb8-47c7-8484-142812fc6d42\") " pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:59:43.287455 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.287439    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 08:59:43.297972 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.297955    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2aee0981-afb8-47c7-8484-142812fc6d42-metrics-certs\") pod \"network-metrics-daemon-c594r\" (UID: \"2aee0981-afb8-47c7-8484-142812fc6d42\") " pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:59:43.385909 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.385875    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:59:43.388042 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.388021    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 08:59:43.398731 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.398694    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e3bc99a4-33c4-4a42-a55c-ef8de46f9502-original-pull-secret\") pod \"global-pull-secret-syncer-z54bd\" (UID: \"e3bc99a4-33c4-4a42-a55c-ef8de46f9502\") " pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:59:43.470887 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.470865    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pn9pp"]
Sep 07 08:59:43.472952 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:43.472929    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba613bd6_8f5b_4edc_8a06_b28348288069.slice/crio-de0ff0e76980842982d0ab98ec9961e8ebf10606c37ba1229a8b2b0fb8256d78 WatchSource:0}: Error finding container de0ff0e76980842982d0ab98ec9961e8ebf10606c37ba1229a8b2b0fb8256d78: Status 404 returned error can't find the container with id de0ff0e76980842982d0ab98ec9961e8ebf10606c37ba1229a8b2b0fb8256d78
Sep 07 08:59:43.486787 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.486767    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hmx8\" (UniqueName: \"kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8\") pod \"network-check-target-sgxq8\" (UID: \"e8f68cb8-c496-4c1e-b79d-081e5fc8d121\") " pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:59:43.490951 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.490349    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hmx8\" (UniqueName: \"kubernetes.io/projected/e8f68cb8-c496-4c1e-b79d-081e5fc8d121-kube-api-access-6hmx8\") pod \"network-check-target-sgxq8\" (UID: \"e8f68cb8-c496-4c1e-b79d-081e5fc8d121\") " pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:59:43.547480 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.547432    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-z54bd"
Sep 07 08:59:43.556304 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.556287    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-txgtt\""
Sep 07 08:59:43.562275 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.562256    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mlzfg\""
Sep 07 08:59:43.565789 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.565704    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c594r"
Sep 07 08:59:43.571310 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.571294    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:59:43.839903 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.839884    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-z54bd"]
Sep 07 08:59:43.842124 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:43.842103    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3bc99a4_33c4_4a42_a55c_ef8de46f9502.slice/crio-16c9a0c8650b6ad10d421fe522cd4787c29ae6c6779c9f00319db10de2904247 WatchSource:0}: Error finding container 16c9a0c8650b6ad10d421fe522cd4787c29ae6c6779c9f00319db10de2904247: Status 404 returned error can't find the container with id 16c9a0c8650b6ad10d421fe522cd4787c29ae6c6779c9f00319db10de2904247
Sep 07 08:59:43.872466 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.872443    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c594r"]
Sep 07 08:59:43.875880 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:43.875856    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aee0981_afb8_47c7_8484_142812fc6d42.slice/crio-fd3e27ae801376fbf30adc35142d4b1dd42d63f16df463c5ba36328ced95c6ff WatchSource:0}: Error finding container fd3e27ae801376fbf30adc35142d4b1dd42d63f16df463c5ba36328ced95c6ff: Status 404 returned error can't find the container with id fd3e27ae801376fbf30adc35142d4b1dd42d63f16df463c5ba36328ced95c6ff
Sep 07 08:59:43.896128 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.896108    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sgxq8"]
Sep 07 08:59:43.898002 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:43.897979    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8f68cb8_c496_4c1e_b79d_081e5fc8d121.slice/crio-8391ed5f07cbb3fee7039245b1f6429a4a7363a570266d07064a4eb87c90af46 WatchSource:0}: Error finding container 8391ed5f07cbb3fee7039245b1f6429a4a7363a570266d07064a4eb87c90af46: Status 404 returned error can't find the container with id 8391ed5f07cbb3fee7039245b1f6429a4a7363a570266d07064a4eb87c90af46
Sep 07 08:59:43.960462 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.960397    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sgxq8" event={"ID":"e8f68cb8-c496-4c1e-b79d-081e5fc8d121","Type":"ContainerStarted","Data":"8391ed5f07cbb3fee7039245b1f6429a4a7363a570266d07064a4eb87c90af46"}
Sep 07 08:59:43.961496 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.961471    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z54bd" event={"ID":"e3bc99a4-33c4-4a42-a55c-ef8de46f9502","Type":"ContainerStarted","Data":"16c9a0c8650b6ad10d421fe522cd4787c29ae6c6779c9f00319db10de2904247"}
Sep 07 08:59:43.962966 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.962944    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pn9pp" event={"ID":"ba613bd6-8f5b-4edc-8a06-b28348288069","Type":"ContainerStarted","Data":"0cb238301927bc8c76ecce832895f748467d16378f3a742718178dac320b7ca0"}
Sep 07 08:59:43.963047 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.962996    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pn9pp" event={"ID":"ba613bd6-8f5b-4edc-8a06-b28348288069","Type":"ContainerStarted","Data":"de0ff0e76980842982d0ab98ec9961e8ebf10606c37ba1229a8b2b0fb8256d78"}
Sep 07 08:59:43.964205 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:43.964185    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c594r" event={"ID":"2aee0981-afb8-47c7-8484-142812fc6d42","Type":"ContainerStarted","Data":"fd3e27ae801376fbf30adc35142d4b1dd42d63f16df463c5ba36328ced95c6ff"}
Sep 07 08:59:44.296070 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.295901    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:44.296070 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.295975    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:44.298384 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.298362    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e215d11-e0ca-493d-a261-35932fbea87c-metrics-tls\") pod \"dns-default-7g5t9\" (UID: \"2e215d11-e0ca-493d-a261-35932fbea87c\") " pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:44.298679 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.298654    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls\") pod \"image-registry-c9bc4456f-xpnlz\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") " pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:44.510467 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.510383    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8sncs\""
Sep 07 08:59:44.518982 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.518953    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:44.527812 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.527746    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hmltn\""
Sep 07 08:59:44.537033 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.536366    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:44.779011 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.778903    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:44.816781 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.816609    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c9bc4456f-xpnlz"]
Sep 07 08:59:44.819543 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:44.819515    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d8372a4_648d_4544_b928_faa61f9b936a.slice/crio-18f0527c480503da159bb0485aa329ab5585777e82e68dd4da4cd00c51ca7ec6 WatchSource:0}: Error finding container 18f0527c480503da159bb0485aa329ab5585777e82e68dd4da4cd00c51ca7ec6: Status 404 returned error can't find the container with id 18f0527c480503da159bb0485aa329ab5585777e82e68dd4da4cd00c51ca7ec6
Sep 07 08:59:44.829260 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.829236    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7g5t9"]
Sep 07 08:59:44.838189 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:44.838165    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e215d11_e0ca_493d_a261_35932fbea87c.slice/crio-b925e44bd645d633da8291126c25d784ad5b6ea46db46e72bee4f45bcaed0b54 WatchSource:0}: Error finding container b925e44bd645d633da8291126c25d784ad5b6ea46db46e72bee4f45bcaed0b54: Status 404 returned error can't find the container with id b925e44bd645d633da8291126c25d784ad5b6ea46db46e72bee4f45bcaed0b54
Sep 07 08:59:44.879032 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.879002    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:44.971700 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.971668    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sgxq8" event={"ID":"e8f68cb8-c496-4c1e-b79d-081e5fc8d121","Type":"ContainerStarted","Data":"b499e8ed6117cd15c337eb2bed0320c7abd9c25d2a03060d3d3739957e7db339"}
Sep 07 08:59:44.971850 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.971763    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 08:59:44.976836 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.976687    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pn9pp" event={"ID":"ba613bd6-8f5b-4edc-8a06-b28348288069","Type":"ContainerStarted","Data":"4d2e400330dc660055ecd2614dce74dab3089cc1a961dfc2e2df9b0ce174c6b4"}
Sep 07 08:59:44.979360 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.979296    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7g5t9" event={"ID":"2e215d11-e0ca-493d-a261-35932fbea87c","Type":"ContainerStarted","Data":"b925e44bd645d633da8291126c25d784ad5b6ea46db46e72bee4f45bcaed0b54"}
Sep 07 08:59:44.982700 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.982674    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c594r" event={"ID":"2aee0981-afb8-47c7-8484-142812fc6d42","Type":"ContainerStarted","Data":"062eb19fc4391e367ebe9e2fb6ac947c0ed02dfadd630720906e730b1532124c"}
Sep 07 08:59:44.987292 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.985251    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz" event={"ID":"5d8372a4-648d-4544-b928-faa61f9b936a","Type":"ContainerStarted","Data":"29868838692e45e7a56ca1ad32f595067a3b5f7d7cbe4b9dfea3e9e9bed19172"}
Sep 07 08:59:44.987292 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.985288    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz" event={"ID":"5d8372a4-648d-4544-b928-faa61f9b936a","Type":"ContainerStarted","Data":"18f0527c480503da159bb0485aa329ab5585777e82e68dd4da4cd00c51ca7ec6"}
Sep 07 08:59:44.987292 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.985816    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 08:59:44.989956 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:44.988911    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sgxq8" podStartSLOduration=66.988894547 podStartE2EDuration="1m6.988894547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:58:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:44.987875445 +0000 UTC m=+67.951791693" watchObservedRunningTime="2026-09-07 08:59:44.988894547 +0000 UTC m=+67.952810768"
Sep 07 08:59:45.993225 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:45.992781    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c594r" event={"ID":"2aee0981-afb8-47c7-8484-142812fc6d42","Type":"ContainerStarted","Data":"b42e101dca30d9e7213a895cc53921379a786e73d9b9b1893922f450ae26c040"}
Sep 07 08:59:46.008598 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:46.008547    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz" podStartSLOduration=47.008529667 podStartE2EDuration="47.008529667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:58:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:45.006931024 +0000 UTC m=+67.970847275" watchObservedRunningTime="2026-09-07 08:59:46.008529667 +0000 UTC m=+68.972445889"
Sep 07 08:59:46.008877 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:46.008841    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-c594r" podStartSLOduration=67.04161994 podStartE2EDuration="1m8.008832177s" totalImagesPullingTime="967.212237ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:46.007060453 +0000 UTC m=+68.970976693" watchObservedRunningTime="2026-09-07 08:59:46.008832177 +0000 UTC m=+68.972748399"
Sep 07 08:59:46.999273 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:46.999175    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pn9pp" event={"ID":"ba613bd6-8f5b-4edc-8a06-b28348288069","Type":"ContainerStarted","Data":"42dd38f76ec94930790516e6363b0d3107d8720b210ffa23114a7c2e8bc48a79"}
Sep 07 08:59:47.001425 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:47.001383    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7g5t9" event={"ID":"2e215d11-e0ca-493d-a261-35932fbea87c","Type":"ContainerStarted","Data":"d215d064738a720995b8f756d0cbf7eaa557e72fe6fef536fec6b8fa9873e380"}
Sep 07 08:59:47.001553 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:47.001438    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7g5t9" event={"ID":"2e215d11-e0ca-493d-a261-35932fbea87c","Type":"ContainerStarted","Data":"534ba5e2acfd3476c3606a7e7c80e561815cbe260e4eaa9aeacb7548fcc0234b"}
Sep 07 08:59:47.001868 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:47.001840    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:47.019598 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:47.019556    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pn9pp" podStartSLOduration=17.621519897 podStartE2EDuration="20.019540747s" totalImagesPullingTime="2.39802085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:59:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:47.017925484 +0000 UTC m=+69.981841725" watchObservedRunningTime="2026-09-07 08:59:47.019540747 +0000 UTC m=+69.983456966"
Sep 07 08:59:47.033595 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:47.033555    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7g5t9" podStartSLOduration=33.174157963 podStartE2EDuration="35.033541667s" totalImagesPullingTime="1.859383704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:47.031951049 +0000 UTC m=+69.995867296" watchObservedRunningTime="2026-09-07 08:59:47.033541667 +0000 UTC m=+69.997457888"
Sep 07 08:59:49.010526 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:49.010470    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-z54bd" event={"ID":"e3bc99a4-33c4-4a42-a55c-ef8de46f9502","Type":"ContainerStarted","Data":"74cafc42fe008925302f23d5df5fa72339fda13dc6febb9a9bcfeaa8f56d73d4"}
Sep 07 08:59:49.024939 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:49.024900    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-z54bd" podStartSLOduration=66.56079897 podStartE2EDuration="1m11.024883829s" totalImagesPullingTime="4.464084859s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 08:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:49.024368182 +0000 UTC m=+71.988284419" watchObservedRunningTime="2026-09-07 08:59:49.024883829 +0000 UTC m=+71.988800049"
Sep 07 08:59:49.169007 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:49.168986    2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:49.169106 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:49.169027    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 08:59:49.169258 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:49.169247    2574 scope.go:117] "RemoveContainer" containerID="21485c42e4e99342694069859dbd909bb3538f06304b53033054ee399e02da50"
Sep 07 08:59:49.169391 ip-10-0-141-81 kubenswrapper[2574]: E0907 08:59:49.169375    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-hm7ht_openshift-console-operator(b61be2b6-3472-4523-95d6-c964c440fe56)\"" pod="openshift-console-operator/console-operator-85b454874b-hm7ht" podUID="b61be2b6-3472-4523-95d6-c964c440fe56"
Sep 07 08:59:50.750883 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:50.750851    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:50.753310 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:50.753281    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f4d4c13f-bc7f-4f46-b32b-906e6136122b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-98mdq\" (UID: \"f4d4c13f-bc7f-4f46-b32b-906e6136122b\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:50.851791 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:50.851759    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:50.851910 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:50.851793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-dbgqp\" (UID: \"2584a748-e024-4375-9635-2a9a21590230\") " pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp"
Sep 07 08:59:50.851910 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:50.851812    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:50.852459 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:50.852434    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-service-ca-bundle\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:50.854182 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:50.854158    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b-metrics-certs\") pod \"router-default-54ff77667d-jgqz5\" (UID: \"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b\") " pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:50.854317 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:50.854292    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2584a748-e024-4375-9635-2a9a21590230-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-dbgqp\" (UID: \"2584a748-e024-4375-9635-2a9a21590230\") " pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp"
Sep 07 08:59:50.946462 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:50.946438    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-65k9n\""
Sep 07 08:59:50.955039 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:50.955022    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"
Sep 07 08:59:51.001457 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:51.001387    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9fp47\""
Sep 07 08:59:51.009989 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:51.009963    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp"
Sep 07 08:59:51.049978 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:51.049954    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xz9dr\""
Sep 07 08:59:51.058905 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:51.058677    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:51.223584 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:51.223563    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq"]
Sep 07 08:59:51.225804 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:51.225783    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d4c13f_bc7f_4f46_b32b_906e6136122b.slice/crio-b429ba07dec44c152af8384e9479da9901a893f67c3f1a446b79e682cf50514d WatchSource:0}: Error finding container b429ba07dec44c152af8384e9479da9901a893f67c3f1a446b79e682cf50514d: Status 404 returned error can't find the container with id b429ba07dec44c152af8384e9479da9901a893f67c3f1a446b79e682cf50514d
Sep 07 08:59:51.282146 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:51.282061    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-dbgqp"]
Sep 07 08:59:51.284425 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:51.284404    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2584a748_e024_4375_9635_2a9a21590230.slice/crio-436d033c705795f744990108211ecebc4e7117e3fb125641cf0483a944350cc1 WatchSource:0}: Error finding container 436d033c705795f744990108211ecebc4e7117e3fb125641cf0483a944350cc1: Status 404 returned error can't find the container with id 436d033c705795f744990108211ecebc4e7117e3fb125641cf0483a944350cc1
Sep 07 08:59:51.331493 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:51.331463    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54ff77667d-jgqz5"]
Sep 07 08:59:51.333787 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:51.333760    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c64f4f0_c3fb_4aed_87a4_b88b17d62a0b.slice/crio-5aff6bb72329e5a2e0f62bc3fcbd0638a433b118e36d354916a5262e06a21684 WatchSource:0}: Error finding container 5aff6bb72329e5a2e0f62bc3fcbd0638a433b118e36d354916a5262e06a21684: Status 404 returned error can't find the container with id 5aff6bb72329e5a2e0f62bc3fcbd0638a433b118e36d354916a5262e06a21684
Sep 07 08:59:52.010342 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.010301    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678686f64d-d7tqr"]
Sep 07 08:59:52.013452 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.013429    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"]
Sep 07 08:59:52.013615 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.013565    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678686f64d-d7tqr"
Sep 07 08:59:52.015701 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.015671    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 07 08:59:52.015701 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.015685    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 07 08:59:52.015919 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.015908    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 07 08:59:52.016027 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.016005    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 07 08:59:52.016142 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.016128    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-hjzh9\""
Sep 07 08:59:52.018150 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.018125    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"]
Sep 07 08:59:52.018355 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.018283    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.020044 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.020025    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 07 08:59:52.020464 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.020387    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 07 08:59:52.020609 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.020587    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 07 08:59:52.021357 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.021331    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678686f64d-d7tqr"]
Sep 07 08:59:52.021475 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.021419    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"
Sep 07 08:59:52.021582 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.021556    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 07 08:59:52.021852 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.021812    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"]
Sep 07 08:59:52.023029 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.022956    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 07 08:59:52.023368 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.023348    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"]
Sep 07 08:59:52.031875 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.031843    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54ff77667d-jgqz5" event={"ID":"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b","Type":"ContainerStarted","Data":"4488d8cfd4fe512e698b07690eb111cd0bd998b5d253366c02128832a1bc35cc"}
Sep 07 08:59:52.031990 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.031895    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54ff77667d-jgqz5" event={"ID":"9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b","Type":"ContainerStarted","Data":"5aff6bb72329e5a2e0f62bc3fcbd0638a433b118e36d354916a5262e06a21684"}
Sep 07 08:59:52.034859 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.034826    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp" event={"ID":"2584a748-e024-4375-9635-2a9a21590230","Type":"ContainerStarted","Data":"436d033c705795f744990108211ecebc4e7117e3fb125641cf0483a944350cc1"}
Sep 07 08:59:52.037322 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.037298    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq" event={"ID":"f4d4c13f-bc7f-4f46-b32b-906e6136122b","Type":"ContainerStarted","Data":"b429ba07dec44c152af8384e9479da9901a893f67c3f1a446b79e682cf50514d"}
Sep 07 08:59:52.059866 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.059830    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:52.062807 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.062789    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:52.090564 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.090511    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-54ff77667d-jgqz5" podStartSLOduration=34.090497954 podStartE2EDuration="34.090497954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:52.088499687 +0000 UTC m=+75.052415910" watchObservedRunningTime="2026-09-07 08:59:52.090497954 +0000 UTC m=+75.054414174"
Sep 07 08:59:52.162107 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.162058    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2164d148-df7d-487f-b63b-d82ac924ffe2-ca\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.162107 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.162103    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e9b0c1a-3e8b-489c-a725-2f6b806bc409-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-678686f64d-d7tqr\" (UID: \"3e9b0c1a-3e8b-489c-a725-2f6b806bc409\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678686f64d-d7tqr"
Sep 07 08:59:52.162332 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.162136    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2164d148-df7d-487f-b63b-d82ac924ffe2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.162332 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.162186    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3f9b3f8-81ea-4506-bb7a-ffb1364aad27-tmp\") pod \"klusterlet-addon-workmgr-6448b7c996-v9frb\" (UID: \"d3f9b3f8-81ea-4506-bb7a-ffb1364aad27\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"
Sep 07 08:59:52.162332 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.162209    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k8lc\" (UniqueName: \"kubernetes.io/projected/2164d148-df7d-487f-b63b-d82ac924ffe2-kube-api-access-8k8lc\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.162332 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.162285    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2164d148-df7d-487f-b63b-d82ac924ffe2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.162538 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.162346    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2164d148-df7d-487f-b63b-d82ac924ffe2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.163052 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.163026    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjvwg\" (UniqueName: \"kubernetes.io/projected/3e9b0c1a-3e8b-489c-a725-2f6b806bc409-kube-api-access-cjvwg\") pod \"managed-serviceaccount-addon-agent-678686f64d-d7tqr\" (UID: \"3e9b0c1a-3e8b-489c-a725-2f6b806bc409\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678686f64d-d7tqr"
Sep 07 08:59:52.163191 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.163063    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d3f9b3f8-81ea-4506-bb7a-ffb1364aad27-klusterlet-config\") pod \"klusterlet-addon-workmgr-6448b7c996-v9frb\" (UID: \"d3f9b3f8-81ea-4506-bb7a-ffb1364aad27\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"
Sep 07 08:59:52.163191 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.163079    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdnpq\" (UniqueName: \"kubernetes.io/projected/d3f9b3f8-81ea-4506-bb7a-ffb1364aad27-kube-api-access-gdnpq\") pod \"klusterlet-addon-workmgr-6448b7c996-v9frb\" (UID: \"d3f9b3f8-81ea-4506-bb7a-ffb1364aad27\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"
Sep 07 08:59:52.163191 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.163120    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2164d148-df7d-487f-b63b-d82ac924ffe2-hub\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.164411 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.164330    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:52.260787 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.260645    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:52.263934 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.263907    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2164d148-df7d-487f-b63b-d82ac924ffe2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.264063 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.263948    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2164d148-df7d-487f-b63b-d82ac924ffe2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.264063 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.263992    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjvwg\" (UniqueName: \"kubernetes.io/projected/3e9b0c1a-3e8b-489c-a725-2f6b806bc409-kube-api-access-cjvwg\") pod \"managed-serviceaccount-addon-agent-678686f64d-d7tqr\" (UID: \"3e9b0c1a-3e8b-489c-a725-2f6b806bc409\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678686f64d-d7tqr"
Sep 07 08:59:52.264063 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.264025    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d3f9b3f8-81ea-4506-bb7a-ffb1364aad27-klusterlet-config\") pod \"klusterlet-addon-workmgr-6448b7c996-v9frb\" (UID: \"d3f9b3f8-81ea-4506-bb7a-ffb1364aad27\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"
Sep 07 08:59:52.264063 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.264053    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdnpq\" (UniqueName: \"kubernetes.io/projected/d3f9b3f8-81ea-4506-bb7a-ffb1364aad27-kube-api-access-gdnpq\") pod \"klusterlet-addon-workmgr-6448b7c996-v9frb\" (UID: \"d3f9b3f8-81ea-4506-bb7a-ffb1364aad27\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"
Sep 07 08:59:52.264267 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.264080    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2164d148-df7d-487f-b63b-d82ac924ffe2-hub\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.264267 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.264104    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2164d148-df7d-487f-b63b-d82ac924ffe2-ca\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.264267 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.264129    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e9b0c1a-3e8b-489c-a725-2f6b806bc409-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-678686f64d-d7tqr\" (UID: \"3e9b0c1a-3e8b-489c-a725-2f6b806bc409\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678686f64d-d7tqr"
Sep 07 08:59:52.264267 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.264159    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2164d148-df7d-487f-b63b-d82ac924ffe2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.264267 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.264181    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3f9b3f8-81ea-4506-bb7a-ffb1364aad27-tmp\") pod \"klusterlet-addon-workmgr-6448b7c996-v9frb\" (UID: \"d3f9b3f8-81ea-4506-bb7a-ffb1364aad27\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"
Sep 07 08:59:52.264267 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.264206    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k8lc\" (UniqueName: \"kubernetes.io/projected/2164d148-df7d-487f-b63b-d82ac924ffe2-kube-api-access-8k8lc\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.264975 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.264642    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2164d148-df7d-487f-b63b-d82ac924ffe2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.264975 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.264930    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3f9b3f8-81ea-4506-bb7a-ffb1364aad27-tmp\") pod \"klusterlet-addon-workmgr-6448b7c996-v9frb\" (UID: \"d3f9b3f8-81ea-4506-bb7a-ffb1364aad27\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"
Sep 07 08:59:52.267499 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.267371    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e9b0c1a-3e8b-489c-a725-2f6b806bc409-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-678686f64d-d7tqr\" (UID: \"3e9b0c1a-3e8b-489c-a725-2f6b806bc409\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678686f64d-d7tqr"
Sep 07 08:59:52.267597 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.267447    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2164d148-df7d-487f-b63b-d82ac924ffe2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.267796 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.267772    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d3f9b3f8-81ea-4506-bb7a-ffb1364aad27-klusterlet-config\") pod \"klusterlet-addon-workmgr-6448b7c996-v9frb\" (UID: \"d3f9b3f8-81ea-4506-bb7a-ffb1364aad27\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"
Sep 07 08:59:52.268491 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.268465    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2164d148-df7d-487f-b63b-d82ac924ffe2-ca\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.268629 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.268610    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2164d148-df7d-487f-b63b-d82ac924ffe2-hub\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.269344 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.269325    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2164d148-df7d-487f-b63b-d82ac924ffe2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.277263 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.277237    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdnpq\" (UniqueName: \"kubernetes.io/projected/d3f9b3f8-81ea-4506-bb7a-ffb1364aad27-kube-api-access-gdnpq\") pod \"klusterlet-addon-workmgr-6448b7c996-v9frb\" (UID: \"d3f9b3f8-81ea-4506-bb7a-ffb1364aad27\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"
Sep 07 08:59:52.280907 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.280886    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k8lc\" (UniqueName: \"kubernetes.io/projected/2164d148-df7d-487f-b63b-d82ac924ffe2-kube-api-access-8k8lc\") pod \"cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm\" (UID: \"2164d148-df7d-487f-b63b-d82ac924ffe2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.282062 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.282039    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjvwg\" (UniqueName: \"kubernetes.io/projected/3e9b0c1a-3e8b-489c-a725-2f6b806bc409-kube-api-access-cjvwg\") pod \"managed-serviceaccount-addon-agent-678686f64d-d7tqr\" (UID: \"3e9b0c1a-3e8b-489c-a725-2f6b806bc409\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678686f64d-d7tqr"
Sep 07 08:59:52.335881 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.335833    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678686f64d-d7tqr"
Sep 07 08:59:52.353838 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.353813    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"
Sep 07 08:59:52.361523 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.361330    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"
Sep 07 08:59:52.392999 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.392934    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:52.477325 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.477286    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:52.541503 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:52.541433    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:53.041872 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:53.041835    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:53.043097 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:53.043078    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-54ff77667d-jgqz5"
Sep 07 08:59:53.092655 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:53.092625    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:53.199929 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:53.199881    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:53.398657 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:53.398577    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678686f64d-d7tqr"]
Sep 07 08:59:53.401642 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:53.401611    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e9b0c1a_3e8b_489c_a725_2f6b806bc409.slice/crio-7d13788fe56f80743549d9d3a41b87da1094609fa3e3c1fcf8cda780a9bf2b8d WatchSource:0}: Error finding container 7d13788fe56f80743549d9d3a41b87da1094609fa3e3c1fcf8cda780a9bf2b8d: Status 404 returned error can't find the container with id 7d13788fe56f80743549d9d3a41b87da1094609fa3e3c1fcf8cda780a9bf2b8d
Sep 07 08:59:53.410019 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:53.409991    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"]
Sep 07 08:59:53.412912 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:53.412889    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f9b3f8_81ea_4506_bb7a_ffb1364aad27.slice/crio-b4cc77cfeca0da36c9b901355a97943b79b91839bd5a53405597bae85b43cd7f WatchSource:0}: Error finding container b4cc77cfeca0da36c9b901355a97943b79b91839bd5a53405597bae85b43cd7f: Status 404 returned error can't find the container with id b4cc77cfeca0da36c9b901355a97943b79b91839bd5a53405597bae85b43cd7f
Sep 07 08:59:53.415992 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:53.415938    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm"]
Sep 07 08:59:53.418407 ip-10-0-141-81 kubenswrapper[2574]: W0907 08:59:53.418385    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2164d148_df7d_487f_b63b_d82ac924ffe2.slice/crio-e0b29b9e6e596c603da11241d14c06cbcdd30fe23138d48c43bf63a35b789227 WatchSource:0}: Error finding container e0b29b9e6e596c603da11241d14c06cbcdd30fe23138d48c43bf63a35b789227: Status 404 returned error can't find the container with id e0b29b9e6e596c603da11241d14c06cbcdd30fe23138d48c43bf63a35b789227
Sep 07 08:59:53.496222 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:53.496195    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:53.598827 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:53.598796    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:53.679381 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:53.679349    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:53.883255 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:53.883218    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:54.048256 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:54.048229    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm" event={"ID":"2164d148-df7d-487f-b63b-d82ac924ffe2","Type":"ContainerStarted","Data":"e0b29b9e6e596c603da11241d14c06cbcdd30fe23138d48c43bf63a35b789227"}
Sep 07 08:59:54.049420 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:54.049399    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb" event={"ID":"d3f9b3f8-81ea-4506-bb7a-ffb1364aad27","Type":"ContainerStarted","Data":"b4cc77cfeca0da36c9b901355a97943b79b91839bd5a53405597bae85b43cd7f"}
Sep 07 08:59:54.051419 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:54.051393    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678686f64d-d7tqr" event={"ID":"3e9b0c1a-3e8b-489c-a725-2f6b806bc409","Type":"ContainerStarted","Data":"7d13788fe56f80743549d9d3a41b87da1094609fa3e3c1fcf8cda780a9bf2b8d"}
Sep 07 08:59:54.053099 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:54.053073    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq" event={"ID":"f4d4c13f-bc7f-4f46-b32b-906e6136122b","Type":"ContainerStarted","Data":"5ca4caf7ecb8942366adb790d51bb5b691e5985185f4449339c26fc87690eba2"}
Sep 07 08:59:54.055011 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:54.054989    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp" event={"ID":"2584a748-e024-4375-9635-2a9a21590230","Type":"ContainerStarted","Data":"56e7ddce739f69dea579b539caff48632d59e22f569d3fa7b08f2104dbf4c765"}
Sep 07 08:59:54.068324 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:54.068274    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-98mdq" podStartSLOduration=34.162831883 podStartE2EDuration="36.068256051s" totalImagesPullingTime="1.905424168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:54.06651288 +0000 UTC m=+77.030429102" watchObservedRunningTime="2026-09-07 08:59:54.068256051 +0000 UTC m=+77.032172273"
Sep 07 08:59:54.079924 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:54.079883    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-dbgqp" podStartSLOduration=34.233576483 podStartE2EDuration="36.079870493s" totalImagesPullingTime="1.84629401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 08:59:54.079153822 +0000 UTC m=+77.043070064" watchObservedRunningTime="2026-09-07 08:59:54.079870493 +0000 UTC m=+77.043786714"
Sep 07 08:59:54.210611 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:54.210536    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:54.263044 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:54.262995    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:58.012900 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:58.012866    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7g5t9"
Sep 07 08:59:58.675252 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:58.675133    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:58.752558 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:58.752524    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 08:59:58.854521 ip-10-0-141-81 kubenswrapper[2574]: I0907 08:59:58.854489    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:01.048229 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:01.048139    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:01.093733 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:01.093673    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm" event={"ID":"2164d148-df7d-487f-b63b-d82ac924ffe2","Type":"ContainerStarted","Data":"89d948dfe1f3c113628995a6d1995df47356db1d5a7a5ab82362cc4bd22a4171"}
Sep 07 09:00:01.093733 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:01.093727    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm" event={"ID":"2164d148-df7d-487f-b63b-d82ac924ffe2","Type":"ContainerStarted","Data":"1b568f4a6237534fb850aeb7c6af6e908395249ca5405db49598eef58233908b"}
Sep 07 09:00:01.093733 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:01.093739    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm" event={"ID":"2164d148-df7d-487f-b63b-d82ac924ffe2","Type":"ContainerStarted","Data":"b808faffce26ed8dab2bb09e65cef0966b759a0edfe5779f2cbc4cd04b72b497"}
Sep 07 09:00:01.095309 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:01.095286    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb" event={"ID":"d3f9b3f8-81ea-4506-bb7a-ffb1364aad27","Type":"ContainerStarted","Data":"0e824292445c5fcd0e37a7a745d988d7c89aaf16674eccd654058654ed7f9b16"}
Sep 07 09:00:01.095437 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:01.095372    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"
Sep 07 09:00:01.097165 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:01.097138    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678686f64d-d7tqr" event={"ID":"3e9b0c1a-3e8b-489c-a725-2f6b806bc409","Type":"ContainerStarted","Data":"1f751abc758bc1df1abb4171f9f6295982a663733980b8c1125d95332e0fa2c9"}
Sep 07 09:00:01.097544 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:01.097527    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb"
Sep 07 09:00:01.111681 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:01.111628    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdb59bb9-tl9rm" podStartSLOduration=2.812953908 podStartE2EDuration="10.111611619s" totalImagesPullingTime="7.298657711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:01.109907508 +0000 UTC m=+84.073823735" watchObservedRunningTime="2026-09-07 09:00:01.111611619 +0000 UTC m=+84.075527880"
Sep 07 09:00:01.124629 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:01.124565    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6448b7c996-v9frb" podStartSLOduration=2.832772054 podStartE2EDuration="10.124545169s" totalImagesPullingTime="7.291773115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:01.124029831 +0000 UTC m=+84.087946072" watchObservedRunningTime="2026-09-07 09:00:01.124545169 +0000 UTC m=+84.088461391"
Sep 07 09:00:01.146699 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:01.146642    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-678686f64d-d7tqr" podStartSLOduration=2.845522121 podStartE2EDuration="10.146627387s" totalImagesPullingTime="7.301105266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:01.145205313 +0000 UTC m=+84.109121536" watchObservedRunningTime="2026-09-07 09:00:01.146627387 +0000 UTC m=+84.110543608"
Sep 07 09:00:01.295375 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:01.295333    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:01.532747 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:01.532687    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:02.536679 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:02.536649    2574 scope.go:117] "RemoveContainer" containerID="21485c42e4e99342694069859dbd909bb3538f06304b53033054ee399e02da50"
Sep 07 09:00:03.091641 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.091604    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nnb6c"]
Sep 07 09:00:03.093923 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.093907    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.096271 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.096243    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 07 09:00:03.098161 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.098138    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 07 09:00:03.098411 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.098389    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jcq2k\""
Sep 07 09:00:03.098498 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.098399    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 07 09:00:03.098681 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.098656    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 07 09:00:03.107944 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.107925    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:00:03.108198 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.108176    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-hm7ht" event={"ID":"b61be2b6-3472-4523-95d6-c964c440fe56","Type":"ContainerStarted","Data":"3547639c684ecf89154c4edd1709f8c04363e94ffc7b84c495c20117297c09e9"}
Sep 07 09:00:03.108517 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.108468    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 09:00:03.111529 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.111499    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:03.140297 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.140234    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-hm7ht" podStartSLOduration=38.428564698 podStartE2EDuration="45.140212632s" totalImagesPullingTime="6.711647934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 08:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:03.138966033 +0000 UTC m=+86.102882280" watchObservedRunningTime="2026-09-07 09:00:03.140212632 +0000 UTC m=+86.104128855"
Sep 07 09:00:03.210534 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.210493    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:03.265857 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.265826    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-tls\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.266058 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.265863    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b40eeca6-4402-403d-9029-79ddd8d0e073-metrics-client-ca\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.266058 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.265904    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-wtmp\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.266058 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.265976    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-accelerators-collector-config\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.266221 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.266084    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b40eeca6-4402-403d-9029-79ddd8d0e073-sys\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.266221 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.266111    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-textfile\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.267179 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.267147    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b40eeca6-4402-403d-9029-79ddd8d0e073-root\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.267263 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.267198    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9v4w\" (UniqueName: \"kubernetes.io/projected/b40eeca6-4402-403d-9029-79ddd8d0e073-kube-api-access-p9v4w\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.267320 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.267292    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.368198 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.368093    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-wtmp\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.368198 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.368156    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-accelerators-collector-config\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.368426 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.368239    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b40eeca6-4402-403d-9029-79ddd8d0e073-sys\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.368426 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.368282    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-textfile\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.368426 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.368317    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b40eeca6-4402-403d-9029-79ddd8d0e073-root\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.368426 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.368339    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9v4w\" (UniqueName: \"kubernetes.io/projected/b40eeca6-4402-403d-9029-79ddd8d0e073-kube-api-access-p9v4w\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.368426 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.368352    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b40eeca6-4402-403d-9029-79ddd8d0e073-sys\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.368426 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.368391    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.368426 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.368417    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b40eeca6-4402-403d-9029-79ddd8d0e073-root\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.368807 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.368426    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-wtmp\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.368807 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.368440    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-tls\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.368807 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.368463    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b40eeca6-4402-403d-9029-79ddd8d0e073-metrics-client-ca\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.368963 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.368833    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-accelerators-collector-config\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.369119 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.369095    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b40eeca6-4402-403d-9029-79ddd8d0e073-metrics-client-ca\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.369198 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.369146    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-textfile\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.371142 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.371115    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.375737 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.375700    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9v4w\" (UniqueName: \"kubernetes.io/projected/b40eeca6-4402-403d-9029-79ddd8d0e073-kube-api-access-p9v4w\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.377014 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.376991    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b40eeca6-4402-403d-9029-79ddd8d0e073-node-exporter-tls\") pod \"node-exporter-nnb6c\" (UID: \"b40eeca6-4402-403d-9029-79ddd8d0e073\") " pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.404771 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.404733    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nnb6c"
Sep 07 09:00:03.425188 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:00:03.425069    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40eeca6_4402_403d_9029_79ddd8d0e073.slice/crio-25ee745a63a338179e357700534bdd295346cd22b770b4fe8f90f07b2f65e1db WatchSource:0}: Error finding container 25ee745a63a338179e357700534bdd295346cd22b770b4fe8f90f07b2f65e1db: Status 404 returned error can't find the container with id 25ee745a63a338179e357700534bdd295346cd22b770b4fe8f90f07b2f65e1db
Sep 07 09:00:03.452843 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.452798    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:03.475364 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.475334    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-hm7ht"
Sep 07 09:00:03.549398 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.549349    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:03.643588 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:03.643505    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:04.014050 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:04.014014    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:04.113735 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:04.113684    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nnb6c" event={"ID":"b40eeca6-4402-403d-9029-79ddd8d0e073","Type":"ContainerStarted","Data":"25ee745a63a338179e357700534bdd295346cd22b770b4fe8f90f07b2f65e1db"}
Sep 07 09:00:04.169262 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:04.169226    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:04.236896 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:04.236842    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:04.325370 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:04.325283    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:04.427196 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:04.427159    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:04.523907 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:04.523868    2574 patch_prober.go:28] interesting pod/image-registry-c9bc4456f-xpnlz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 07 09:00:04.524074 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:04.523932    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz" podUID="5d8372a4-648d-4544-b928-faa61f9b936a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 09:00:04.579844 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:04.579757    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:04.730988 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:04.730940    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:04.930267 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:04.930230    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:05.031188 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:05.031153    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:05.546334 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:05.546285    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:06.127940 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:06.127906    2574 generic.go:356] "Generic (PLEG): container finished" podID="b40eeca6-4402-403d-9029-79ddd8d0e073" containerID="a61d51a9712672f5459f44bcbafc3c3d3a014067cfa7a8591f22ee71185de5f4" exitCode=0
Sep 07 09:00:06.128337 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:06.127959    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nnb6c" event={"ID":"b40eeca6-4402-403d-9029-79ddd8d0e073","Type":"ContainerDied","Data":"a61d51a9712672f5459f44bcbafc3c3d3a014067cfa7a8591f22ee71185de5f4"}
Sep 07 09:00:06.204643 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:06.204617    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:06.284194 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:06.284166    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:06.786280 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:06.786223    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:06.955312 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:06.955239    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:07.007463 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:07.007431    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 09:00:07.030588 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:07.030559    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:07.134043 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:07.133940    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nnb6c" event={"ID":"b40eeca6-4402-403d-9029-79ddd8d0e073","Type":"ContainerStarted","Data":"4db62bd59080664c9411a842a81b80e79362426b2fe26f4fd346a012aec5a2ea"}
Sep 07 09:00:07.134468 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:07.134062    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nnb6c" event={"ID":"b40eeca6-4402-403d-9029-79ddd8d0e073","Type":"ContainerStarted","Data":"86cb3188d65ef56aa7d13f0967298bad98498429a46618e543fbe96fb3308016"}
Sep 07 09:00:07.134647 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:07.134624    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:07.153328 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:07.153273    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nnb6c" podStartSLOduration=2.054172857 podStartE2EDuration="4.153256193s" totalImagesPullingTime="2.026812748s" totalInitContainerRuntime="72.270588ms" isStatefulPod=true podCreationTimestamp="2026-09-07 09:00:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:00:07.15198605 +0000 UTC m=+90.115902316" watchObservedRunningTime="2026-09-07 09:00:07.153256193 +0000 UTC m=+90.117172413"
Sep 07 09:00:07.247331 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:07.247294    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:08.710282 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:08.710251    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:08.819635 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:08.819594    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:09.427797 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:09.427763    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:10.401304 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:10.401274    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:10.502030 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:10.502002    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:12.025023 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:12.024982    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:12.088654 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:12.088619    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:14.822522 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:14.822490    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c9bc4456f-xpnlz"]
Sep 07 09:00:14.892244 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:14.892208    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:14.951048 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:14.951005    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:15.069091 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:15.069049    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:15.121371 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:15.121287    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:15.995536 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:15.995506    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sgxq8"
Sep 07 09:00:16.143264 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:16.143178    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:20.189098 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:20.189063    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:20.242319 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:20.242287    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:20.560226 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:20.560126    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:20.659439 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:20.659400    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:22.961591 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:22.961552    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:23.063020 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:23.062989    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:23.162599 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:23.162561    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:23.262950 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:23.262864    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:29.258634 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:29.258600    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:29.318006 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:29.317972    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:29.711186 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:29.711134    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:29.811564 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:29.811530    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:34.154780 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:34.154743    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:35.606738 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:35.606694    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:35.705086 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:35.705054    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:35.971386 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:35.971356    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:36.027491 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:36.027461    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:36.372204 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:36.372132    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:36.423089 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:36.423056    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:39.842926 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:39.842878    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz" podUID="5d8372a4-648d-4544-b928-faa61f9b936a" containerName="registry" containerID="cri-o://29868838692e45e7a56ca1ad32f595067a3b5f7d7cbe4b9dfea3e9e9bed19172" gracePeriod=30
Sep 07 09:00:40.078272 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.078252    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 09:00:40.152463 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.152399    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d8372a4-648d-4544-b928-faa61f9b936a-registry-certificates\") pod \"5d8372a4-648d-4544-b928-faa61f9b936a\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") "
Sep 07 09:00:40.152463 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.152433    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls\") pod \"5d8372a4-648d-4544-b928-faa61f9b936a\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") "
Sep 07 09:00:40.152463 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.152460    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d8372a4-648d-4544-b928-faa61f9b936a-image-registry-private-configuration\") pod \"5d8372a4-648d-4544-b928-faa61f9b936a\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") "
Sep 07 09:00:40.152689 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.152489    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-bound-sa-token\") pod \"5d8372a4-648d-4544-b928-faa61f9b936a\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") "
Sep 07 09:00:40.152689 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.152529    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d8372a4-648d-4544-b928-faa61f9b936a-ca-trust-extracted\") pod \"5d8372a4-648d-4544-b928-faa61f9b936a\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") "
Sep 07 09:00:40.152689 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.152574    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9d7n\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-kube-api-access-d9d7n\") pod \"5d8372a4-648d-4544-b928-faa61f9b936a\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") "
Sep 07 09:00:40.152689 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.152619    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d8372a4-648d-4544-b928-faa61f9b936a-installation-pull-secrets\") pod \"5d8372a4-648d-4544-b928-faa61f9b936a\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") "
Sep 07 09:00:40.152689 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.152669    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d8372a4-648d-4544-b928-faa61f9b936a-trusted-ca\") pod \"5d8372a4-648d-4544-b928-faa61f9b936a\" (UID: \"5d8372a4-648d-4544-b928-faa61f9b936a\") "
Sep 07 09:00:40.152997 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.152968    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d8372a4-648d-4544-b928-faa61f9b936a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5d8372a4-648d-4544-b928-faa61f9b936a" (UID: "5d8372a4-648d-4544-b928-faa61f9b936a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:00:40.153305 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.153280    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d8372a4-648d-4544-b928-faa61f9b936a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5d8372a4-648d-4544-b928-faa61f9b936a" (UID: "5d8372a4-648d-4544-b928-faa61f9b936a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:00:40.155197 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.155150    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d8372a4-648d-4544-b928-faa61f9b936a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5d8372a4-648d-4544-b928-faa61f9b936a" (UID: "5d8372a4-648d-4544-b928-faa61f9b936a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:00:40.155301 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.155218    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5d8372a4-648d-4544-b928-faa61f9b936a" (UID: "5d8372a4-648d-4544-b928-faa61f9b936a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:00:40.155558 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.155531    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d8372a4-648d-4544-b928-faa61f9b936a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5d8372a4-648d-4544-b928-faa61f9b936a" (UID: "5d8372a4-648d-4544-b928-faa61f9b936a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:00:40.155654 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.155584    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5d8372a4-648d-4544-b928-faa61f9b936a" (UID: "5d8372a4-648d-4544-b928-faa61f9b936a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:00:40.155694 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.155644    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-kube-api-access-d9d7n" (OuterVolumeSpecName: "kube-api-access-d9d7n") pod "5d8372a4-648d-4544-b928-faa61f9b936a" (UID: "5d8372a4-648d-4544-b928-faa61f9b936a"). InnerVolumeSpecName "kube-api-access-d9d7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:00:40.161510 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.161486    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d8372a4-648d-4544-b928-faa61f9b936a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5d8372a4-648d-4544-b928-faa61f9b936a" (UID: "5d8372a4-648d-4544-b928-faa61f9b936a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:00:40.254016 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.253992    2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5d8372a4-648d-4544-b928-faa61f9b936a-installation-pull-secrets\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:00:40.254016 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.254015    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5d8372a4-648d-4544-b928-faa61f9b936a-trusted-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:00:40.254158 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.254028    2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5d8372a4-648d-4544-b928-faa61f9b936a-registry-certificates\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:00:40.254158 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.254042    2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-registry-tls\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:00:40.254158 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.254054    2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5d8372a4-648d-4544-b928-faa61f9b936a-image-registry-private-configuration\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:00:40.254158 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.254067    2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-bound-sa-token\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:00:40.254158 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.254079    2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5d8372a4-648d-4544-b928-faa61f9b936a-ca-trust-extracted\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:00:40.254158 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.254091    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9d7n\" (UniqueName: \"kubernetes.io/projected/5d8372a4-648d-4544-b928-faa61f9b936a-kube-api-access-d9d7n\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:00:40.262253 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.262227    2574 generic.go:356] "Generic (PLEG): container finished" podID="5d8372a4-648d-4544-b928-faa61f9b936a" containerID="29868838692e45e7a56ca1ad32f595067a3b5f7d7cbe4b9dfea3e9e9bed19172" exitCode=0
Sep 07 09:00:40.262351 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.262269    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz" event={"ID":"5d8372a4-648d-4544-b928-faa61f9b936a","Type":"ContainerDied","Data":"29868838692e45e7a56ca1ad32f595067a3b5f7d7cbe4b9dfea3e9e9bed19172"}
Sep 07 09:00:40.262351 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.262314    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz" event={"ID":"5d8372a4-648d-4544-b928-faa61f9b936a","Type":"ContainerDied","Data":"18f0527c480503da159bb0485aa329ab5585777e82e68dd4da4cd00c51ca7ec6"}
Sep 07 09:00:40.262351 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.262342    2574 scope.go:117] "RemoveContainer" containerID="29868838692e45e7a56ca1ad32f595067a3b5f7d7cbe4b9dfea3e9e9bed19172"
Sep 07 09:00:40.262502 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.262273    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c9bc4456f-xpnlz"
Sep 07 09:00:40.272367 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.272352    2574 scope.go:117] "RemoveContainer" containerID="29868838692e45e7a56ca1ad32f595067a3b5f7d7cbe4b9dfea3e9e9bed19172"
Sep 07 09:00:40.272804 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:00:40.272783    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29868838692e45e7a56ca1ad32f595067a3b5f7d7cbe4b9dfea3e9e9bed19172\": container with ID starting with 29868838692e45e7a56ca1ad32f595067a3b5f7d7cbe4b9dfea3e9e9bed19172 not found: ID does not exist" containerID="29868838692e45e7a56ca1ad32f595067a3b5f7d7cbe4b9dfea3e9e9bed19172"
Sep 07 09:00:40.272874 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.272812    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29868838692e45e7a56ca1ad32f595067a3b5f7d7cbe4b9dfea3e9e9bed19172"} err="failed to get container status \"29868838692e45e7a56ca1ad32f595067a3b5f7d7cbe4b9dfea3e9e9bed19172\": rpc error: code = NotFound desc = could not find container \"29868838692e45e7a56ca1ad32f595067a3b5f7d7cbe4b9dfea3e9e9bed19172\": container with ID starting with 29868838692e45e7a56ca1ad32f595067a3b5f7d7cbe4b9dfea3e9e9bed19172 not found: ID does not exist"
Sep 07 09:00:40.282088 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.282068    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c9bc4456f-xpnlz"]
Sep 07 09:00:40.285579 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:40.285559    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-c9bc4456f-xpnlz"]
Sep 07 09:00:41.541399 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:41.541361    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d8372a4-648d-4544-b928-faa61f9b936a" path="/var/lib/kubelet/pods/5d8372a4-648d-4544-b928-faa61f9b936a/volumes"
Sep 07 09:00:41.931910 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:41.931878    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:41.981994 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:41.981963    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:42.272851 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:42.272771    2574 generic.go:356] "Generic (PLEG): container finished" podID="930228a4-40c6-4a1b-a2a1-599f1de15ec3" containerID="5b17449038c84e9614420aebb077bb800c90a73cf43e0da4a1d2d36ad84a1f69" exitCode=0
Sep 07 09:00:42.272851 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:42.272817    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx" event={"ID":"930228a4-40c6-4a1b-a2a1-599f1de15ec3","Type":"ContainerDied","Data":"5b17449038c84e9614420aebb077bb800c90a73cf43e0da4a1d2d36ad84a1f69"}
Sep 07 09:00:42.273135 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:42.273122    2574 scope.go:117] "RemoveContainer" containerID="5b17449038c84e9614420aebb077bb800c90a73cf43e0da4a1d2d36ad84a1f69"
Sep 07 09:00:42.886036 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:42.886005    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54ff77667d-jgqz5_9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b/router/0.log"
Sep 07 09:00:42.902395 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:42.902365    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hzt78_c82b73b1-7d84-4d8f-be08-c7cf655fe39d/serve-healthcheck-canary/0.log"
Sep 07 09:00:43.279000 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:43.278966    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-4ggpx" event={"ID":"930228a4-40c6-4a1b-a2a1-599f1de15ec3","Type":"ContainerStarted","Data":"679b44822531219474fefaf1c8912597f5f0641b04a6b2f77720e4dbcd51b5b7"}
Sep 07 09:00:44.715142 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:44.715104    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:44.768179 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:44.768142    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:51.327393 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:51.327357    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:51.428031 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:51.428003    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:52.368354 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:52.368316    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:52.470329 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:52.470296    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:52.532881 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:52.532849    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:00:57.340334 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:57.340301    2574 generic.go:356] "Generic (PLEG): container finished" podID="f96ca98e-8ab7-4335-b49f-c97435d6eab7" containerID="c1d1ababcda362681004aeda175aca644adf5c14ad525be653f68afa8f77c04e" exitCode=0
Sep 07 09:00:57.340660 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:57.340381    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4" event={"ID":"f96ca98e-8ab7-4335-b49f-c97435d6eab7","Type":"ContainerDied","Data":"c1d1ababcda362681004aeda175aca644adf5c14ad525be653f68afa8f77c04e"}
Sep 07 09:00:57.340705 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:57.340662    2574 scope.go:117] "RemoveContainer" containerID="c1d1ababcda362681004aeda175aca644adf5c14ad525be653f68afa8f77c04e"
Sep 07 09:00:58.345871 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:58.345837    2574 generic.go:356] "Generic (PLEG): container finished" podID="28768fce-452e-43c7-b924-4093e574e45a" containerID="fb92cfd974e0b26d5f53e5ef37621f4171fa5ca110485373a0f9df779146da39" exitCode=0
Sep 07 09:00:58.346348 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:58.345920    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754" event={"ID":"28768fce-452e-43c7-b924-4093e574e45a","Type":"ContainerDied","Data":"fb92cfd974e0b26d5f53e5ef37621f4171fa5ca110485373a0f9df779146da39"}
Sep 07 09:00:58.346348 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:58.346320    2574 scope.go:117] "RemoveContainer" containerID="fb92cfd974e0b26d5f53e5ef37621f4171fa5ca110485373a0f9df779146da39"
Sep 07 09:00:58.347794 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:58.347776    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-cmlg4" event={"ID":"f96ca98e-8ab7-4335-b49f-c97435d6eab7","Type":"ContainerStarted","Data":"02f841397deffa3f992a3f02b5412250409185c94c006f0bb2d2ba3ced4e26ba"}
Sep 07 09:00:59.353431 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:00:59.353395    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-wg754" event={"ID":"28768fce-452e-43c7-b924-4093e574e45a","Type":"ContainerStarted","Data":"b5f676b85779a684cb7af4af5b832719d28d65a92ec2f1b0af3aa12f9974cb38"}
Sep 07 09:01:11.929355 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:01:11.929322    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:01:22.341813 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:01:22.341773    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:01:22.542939 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:01:22.542905    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:01:32.798758 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:01:32.798664    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:01:32.940849 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:01:32.940814    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:01:33.046485 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:01:33.046449    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:01:33.533773 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:01:33.533738    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:01:33.635537 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:01:33.635458    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:01:42.086699 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:01:42.086662    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:01:43.825628 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:01:43.825571    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:01:43.876318 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:01:43.876280    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:01:43.925909 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:01:43.925865    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:01:43.979659 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:01:43.979620    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:02:13.538182 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:02:13.538144    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:02:13.602318 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:02:13.602290    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:02:37.556577 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:02:37.556542    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:02:37.608724 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:02:37.608694    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:02:41.421912 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:02:41.421868    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:02:41.481860 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:02:41.481828    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:02:49.586029 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:02:49.585997    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:02:49.714178 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:02:49.714150    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:02:52.074643 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:02:52.074597    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:02:52.178410 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:02:52.178378    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:02:52.226553 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:02:52.226521    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:02:59.720588 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:02:59.720509    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:02:59.820600 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:02:59.820561    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:02:59.909434 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:02:59.909399    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:03:25.019356 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:25.019319    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:03:25.120410 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:25.120371    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:03:37.441478 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:37.441450    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:03:37.447852 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:37.447828    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:03:37.449278 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:37.449257    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:03:37.452377 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:37.452357    2574 kubelet.go:1648] "Image garbage collection succeeded"
Sep 07 09:03:37.456459 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:37.456441    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:03:46.381156 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:46.381121    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:03:47.860381 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:47.860348    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:03:49.969527 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:49.969486    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:03:50.070117 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:50.070081    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:03:51.460863 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:51.460832    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:03:51.564154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:51.564121    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:03:51.668067 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:51.668037    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:03:53.822334 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:53.822307    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:03:54.016410 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:54.016378    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:03:56.762781 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:56.762751    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:03:56.815035 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:03:56.815014    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:01.429111 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:01.429080    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:04.245452 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:04.245415    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:04.345183 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:04.345151    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:05.290453 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:05.290407    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:05.391175 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:05.391139    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:09.994145 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:09.994105    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:10.080234 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:10.080201    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:12.193796 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:12.193763    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:27.397867 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:27.397823    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:27.491592 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:27.491561    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:28.002654 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.002622    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:28.189367 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.189327    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:28.295340 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.295263    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:28.342134 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.342090    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"]
Sep 07 09:04:28.342391 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.342379    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d8372a4-648d-4544-b928-faa61f9b936a" containerName="registry"
Sep 07 09:04:28.342432 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.342393    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d8372a4-648d-4544-b928-faa61f9b936a" containerName="registry"
Sep 07 09:04:28.342464 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.342443    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d8372a4-648d-4544-b928-faa61f9b936a" containerName="registry"
Sep 07 09:04:28.344494 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.344477    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.347530 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.347508    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 07 09:04:28.347636 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.347583    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 07 09:04:28.347751 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.347694    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 07 09:04:28.347957 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.347933    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 07 09:04:28.348070 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.347767    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-c5k27\""
Sep 07 09:04:28.366374 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.366354    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"]
Sep 07 09:04:28.438306 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.438231    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hx2f\" (UniqueName: \"kubernetes.io/projected/c077dde0-13cf-4351-a65a-14cb61f02fec-kube-api-access-6hx2f\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.438306 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.438272    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c077dde0-13cf-4351-a65a-14cb61f02fec-webhook-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.438780 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.438316    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c077dde0-13cf-4351-a65a-14cb61f02fec-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.438780 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.438344    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c077dde0-13cf-4351-a65a-14cb61f02fec-tmp\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.438780 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.438373    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c077dde0-13cf-4351-a65a-14cb61f02fec-manifests\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.532874 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.532840    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:28.538950 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.538929    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c077dde0-13cf-4351-a65a-14cb61f02fec-webhook-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.539041 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.538966    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c077dde0-13cf-4351-a65a-14cb61f02fec-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.539041 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.538987    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c077dde0-13cf-4351-a65a-14cb61f02fec-tmp\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.539041 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.539006    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c077dde0-13cf-4351-a65a-14cb61f02fec-manifests\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.539182 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.539133    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hx2f\" (UniqueName: \"kubernetes.io/projected/c077dde0-13cf-4351-a65a-14cb61f02fec-kube-api-access-6hx2f\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.539385 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.539365    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c077dde0-13cf-4351-a65a-14cb61f02fec-manifests\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.541305 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.541277    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c077dde0-13cf-4351-a65a-14cb61f02fec-tmp\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.541472 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.541453    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c077dde0-13cf-4351-a65a-14cb61f02fec-webhook-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.541618 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.541596    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c077dde0-13cf-4351-a65a-14cb61f02fec-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.555260 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.555215    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hx2f\" (UniqueName: \"kubernetes.io/projected/c077dde0-13cf-4351-a65a-14cb61f02fec-kube-api-access-6hx2f\") pod \"opendatahub-operator-controller-manager-78c59b5944-z2bls\" (UID: \"c077dde0-13cf-4351-a65a-14cb61f02fec\") " pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.632759 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.632732    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:28.654817 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.654793    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:28.707279 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.707253    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:28.808639 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.808549    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:28.924933 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.924897    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:28.953802 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.953780    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"]
Sep 07 09:04:28.957237 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:04:28.957215    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc077dde0_13cf_4351_a65a_14cb61f02fec.slice/crio-60d50e8a7bf15a9c3c66b3324a48dc0650a03cafa0b29d295feabe2c292c7207 WatchSource:0}: Error finding container 60d50e8a7bf15a9c3c66b3324a48dc0650a03cafa0b29d295feabe2c292c7207: Status 404 returned error can't find the container with id 60d50e8a7bf15a9c3c66b3324a48dc0650a03cafa0b29d295feabe2c292c7207
Sep 07 09:04:28.958962 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:28.958942    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 09:04:29.156428 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:29.156403    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:29.168981 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:29.168956    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls" event={"ID":"c077dde0-13cf-4351-a65a-14cb61f02fec","Type":"ContainerStarted","Data":"60d50e8a7bf15a9c3c66b3324a48dc0650a03cafa0b29d295feabe2c292c7207"}
Sep 07 09:04:29.257174 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:29.257143    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:32.186544 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:32.186506    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls" event={"ID":"c077dde0-13cf-4351-a65a-14cb61f02fec","Type":"ContainerStarted","Data":"80d38051f2d27ecea0eed910212f4905f3f8536ebdb0335ad713b200905db0a6"}
Sep 07 09:04:33.192363 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:33.192334    2574 generic.go:356] "Generic (PLEG): container finished" podID="c077dde0-13cf-4351-a65a-14cb61f02fec" containerID="80d38051f2d27ecea0eed910212f4905f3f8536ebdb0335ad713b200905db0a6" exitCode=0
Sep 07 09:04:33.192747 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:33.192407    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls" event={"ID":"c077dde0-13cf-4351-a65a-14cb61f02fec","Type":"ContainerDied","Data":"80d38051f2d27ecea0eed910212f4905f3f8536ebdb0335ad713b200905db0a6"}
Sep 07 09:04:34.199147 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:34.199107    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls" event={"ID":"c077dde0-13cf-4351-a65a-14cb61f02fec","Type":"ContainerStarted","Data":"14de667ec0e5d181aa39fef1d8e1a0574a608ff5ad19cf9777175f3a2e1c9677"}
Sep 07 09:04:34.199554 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:34.199218    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:34.219281 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:34.219238    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls" podStartSLOduration=1.952710035 podStartE2EDuration="6.219223038s" totalImagesPullingTime="3.257445143s" totalInitContainerRuntime="1.00906786s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:04:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:04:34.217450802 +0000 UTC m=+357.181367036" watchObservedRunningTime="2026-09-07 09:04:34.219223038 +0000 UTC m=+357.183139259"
Sep 07 09:04:45.205446 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:45.205414    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78c59b5944-z2bls"
Sep 07 09:04:48.554104 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.554070    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:48.667657 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.667624    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:48.844433 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.844356    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"]
Sep 07 09:04:48.856065 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.856043    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"
Sep 07 09:04:48.856809 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.856788    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"]
Sep 07 09:04:48.858245 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.858083    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 07 09:04:48.858245 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.858138    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 07 09:04:48.865780 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.865705    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:48.889516 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.889490    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ec5585a-b1fb-4a96-b88a-22dcbaa6baad-tls-certs\") pod \"kube-auth-proxy-5b86566b9f-kmwb2\" (UID: \"9ec5585a-b1fb-4a96-b88a-22dcbaa6baad\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"
Sep 07 09:04:48.889652 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.889553    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lktbh\" (UniqueName: \"kubernetes.io/projected/9ec5585a-b1fb-4a96-b88a-22dcbaa6baad-kube-api-access-lktbh\") pod \"kube-auth-proxy-5b86566b9f-kmwb2\" (UID: \"9ec5585a-b1fb-4a96-b88a-22dcbaa6baad\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"
Sep 07 09:04:48.889652 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.889624    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ec5585a-b1fb-4a96-b88a-22dcbaa6baad-tmp\") pod \"kube-auth-proxy-5b86566b9f-kmwb2\" (UID: \"9ec5585a-b1fb-4a96-b88a-22dcbaa6baad\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"
Sep 07 09:04:48.990923 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.990891    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ec5585a-b1fb-4a96-b88a-22dcbaa6baad-tls-certs\") pod \"kube-auth-proxy-5b86566b9f-kmwb2\" (UID: \"9ec5585a-b1fb-4a96-b88a-22dcbaa6baad\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"
Sep 07 09:04:48.991086 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.990961    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lktbh\" (UniqueName: \"kubernetes.io/projected/9ec5585a-b1fb-4a96-b88a-22dcbaa6baad-kube-api-access-lktbh\") pod \"kube-auth-proxy-5b86566b9f-kmwb2\" (UID: \"9ec5585a-b1fb-4a96-b88a-22dcbaa6baad\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"
Sep 07 09:04:48.991086 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.990993    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ec5585a-b1fb-4a96-b88a-22dcbaa6baad-tmp\") pod \"kube-auth-proxy-5b86566b9f-kmwb2\" (UID: \"9ec5585a-b1fb-4a96-b88a-22dcbaa6baad\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"
Sep 07 09:04:48.991086 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:04:48.991058    2574 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 07 09:04:48.991249 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:04:48.991126    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ec5585a-b1fb-4a96-b88a-22dcbaa6baad-tls-certs podName:9ec5585a-b1fb-4a96-b88a-22dcbaa6baad nodeName:}" failed. No retries permitted until 2026-09-07 09:04:49.491109932 +0000 UTC m=+372.455026147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9ec5585a-b1fb-4a96-b88a-22dcbaa6baad-tls-certs") pod "kube-auth-proxy-5b86566b9f-kmwb2" (UID: "9ec5585a-b1fb-4a96-b88a-22dcbaa6baad") : secret "kube-auth-proxy-tls" not found
Sep 07 09:04:48.993320 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:48.993299    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ec5585a-b1fb-4a96-b88a-22dcbaa6baad-tmp\") pod \"kube-auth-proxy-5b86566b9f-kmwb2\" (UID: \"9ec5585a-b1fb-4a96-b88a-22dcbaa6baad\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"
Sep 07 09:04:49.019478 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:49.019445    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lktbh\" (UniqueName: \"kubernetes.io/projected/9ec5585a-b1fb-4a96-b88a-22dcbaa6baad-kube-api-access-lktbh\") pod \"kube-auth-proxy-5b86566b9f-kmwb2\" (UID: \"9ec5585a-b1fb-4a96-b88a-22dcbaa6baad\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"
Sep 07 09:04:49.083858 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:49.083823    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:49.495773 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:49.495740    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ec5585a-b1fb-4a96-b88a-22dcbaa6baad-tls-certs\") pod \"kube-auth-proxy-5b86566b9f-kmwb2\" (UID: \"9ec5585a-b1fb-4a96-b88a-22dcbaa6baad\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"
Sep 07 09:04:49.498245 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:49.498222    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ec5585a-b1fb-4a96-b88a-22dcbaa6baad-tls-certs\") pod \"kube-auth-proxy-5b86566b9f-kmwb2\" (UID: \"9ec5585a-b1fb-4a96-b88a-22dcbaa6baad\") " pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"
Sep 07 09:04:49.768925 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:49.768853    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"
Sep 07 09:04:50.045884 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.045806    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2"]
Sep 07 09:04:50.049122 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:04:50.049095    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ec5585a_b1fb_4a96_b88a_22dcbaa6baad.slice/crio-63e784c8ae1ffbce434829b18edd8a3eb9de0febe942c7a6392b006d2b34d511 WatchSource:0}: Error finding container 63e784c8ae1ffbce434829b18edd8a3eb9de0febe942c7a6392b006d2b34d511: Status 404 returned error can't find the container with id 63e784c8ae1ffbce434829b18edd8a3eb9de0febe942c7a6392b006d2b34d511
Sep 07 09:04:50.140035 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.140006    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"]
Sep 07 09:04:50.142619 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.142599    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.144492 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.144468    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 07 09:04:50.144609 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.144496    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ccv6f\""
Sep 07 09:04:50.144679 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.144632    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 07 09:04:50.144821 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.144705    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 07 09:04:50.144869 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.144821    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 07 09:04:50.156553 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.156530    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"]
Sep 07 09:04:50.166255 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.166093    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:50.202640 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.202619    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.202815 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.202647    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/223c8f38-0d96-4145-aec4-57b8c4148e57-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.202815 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.202668    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn9lv\" (UniqueName: \"kubernetes.io/projected/223c8f38-0d96-4145-aec4-57b8c4148e57-kube-api-access-fn9lv\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.202815 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.202697    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.202815 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.202794    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.203006 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.202833    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.203006 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.202882    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.231332 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.231296    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:50.271177 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.271152    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2" event={"ID":"9ec5585a-b1fb-4a96-b88a-22dcbaa6baad","Type":"ContainerStarted","Data":"63e784c8ae1ffbce434829b18edd8a3eb9de0febe942c7a6392b006d2b34d511"}
Sep 07 09:04:50.303741 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.303646    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.303741 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.303692    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.303912 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.303747    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.303912 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.303782    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.303912 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.303830    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.303912 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.303870    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/223c8f38-0d96-4145-aec4-57b8c4148e57-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.303912 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.303897    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn9lv\" (UniqueName: \"kubernetes.io/projected/223c8f38-0d96-4145-aec4-57b8c4148e57-kube-api-access-fn9lv\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.304359 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.304332    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.306263 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.306233    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/223c8f38-0d96-4145-aec4-57b8c4148e57-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.306389 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.306360    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.306591 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.306573    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.306695 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.306676    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.311294 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.311264    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn9lv\" (UniqueName: \"kubernetes.io/projected/223c8f38-0d96-4145-aec4-57b8c4148e57-kube-api-access-fn9lv\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.311679 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.311662    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bnpdc\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.452048 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.452013    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:50.725250 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:50.725223    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"]
Sep 07 09:04:50.727488 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:04:50.727458    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod223c8f38_0d96_4145_aec4_57b8c4148e57.slice/crio-33308260930b35ef92a39d82fe514591d1f6041e351a62600464ea64445fbfd6 WatchSource:0}: Error finding container 33308260930b35ef92a39d82fe514591d1f6041e351a62600464ea64445fbfd6: Status 404 returned error can't find the container with id 33308260930b35ef92a39d82fe514591d1f6041e351a62600464ea64445fbfd6
Sep 07 09:04:51.108912 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:51.108829    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:51.278747 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:51.278689    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc" event={"ID":"223c8f38-0d96-4145-aec4-57b8c4148e57","Type":"ContainerStarted","Data":"33308260930b35ef92a39d82fe514591d1f6041e351a62600464ea64445fbfd6"}
Sep 07 09:04:53.290220 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:53.290180    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2" event={"ID":"9ec5585a-b1fb-4a96-b88a-22dcbaa6baad","Type":"ContainerStarted","Data":"4ec8105698ed5b2cb55df453a846760a94b5fb9a6f8b4f2e61bea02b5f9720c3"}
Sep 07 09:04:53.305431 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:53.305374    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b86566b9f-kmwb2" podStartSLOduration=2.739159318 podStartE2EDuration="5.305357956s" totalImagesPullingTime="2.566198638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:04:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:04:53.304157819 +0000 UTC m=+376.268074039" watchObservedRunningTime="2026-09-07 09:04:53.305357956 +0000 UTC m=+376.269274220"
Sep 07 09:04:54.390159 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:54.390124    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 07 09:04:54.390395 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:54.390193    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 07 09:04:54.907613 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:54.907576    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:55.007007 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:55.006976    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:55.092776 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:55.092700    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:55.176068 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:55.175988    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:55.301038 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:55.301003    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc" event={"ID":"223c8f38-0d96-4145-aec4-57b8c4148e57","Type":"ContainerStarted","Data":"8b99413031d57dd6697569836411edae0e330ee4949269fe9072998e3a5de0a4"}
Sep 07 09:04:55.301215 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:55.301202    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:55.302843 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:55.302820    2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-bnpdc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 09:04:55.302965 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:55.302865    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc" podUID="223c8f38-0d96-4145-aec4-57b8c4148e57" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 09:04:55.319395 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:55.319344    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc" podStartSLOduration=1.6596923449999998 podStartE2EDuration="5.319323792s" totalImagesPullingTime="3.659631447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:04:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:04:55.319108243 +0000 UTC m=+378.283024491" watchObservedRunningTime="2026-09-07 09:04:55.319323792 +0000 UTC m=+378.283240017"
Sep 07 09:04:55.354817 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:55.354794    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:56.306181 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:56.306151    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:04:59.353515 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:59.353475    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:59.454435 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:59.454400    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:59.703810 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:59.703775    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:04:59.800592 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:04:59.800563    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:00.005138 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:00.005048    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:06.792157 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:06.792115    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:08.347566 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:08.347531    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:08.402945 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:08.402914    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rw26k"]
Sep 07 09:05:08.407437 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:08.407414    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rw26k"
Sep 07 09:05:08.412351 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:08.412332    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-xxms7\""
Sep 07 09:05:08.413533 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:08.413512    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 07 09:05:08.441915 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:08.441892    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rw26k"]
Sep 07 09:05:08.454943 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:08.454918    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16652170-c20c-4344-9e2c-55d0f861edc2-cert\") pod \"kserve-controller-manager-d6f4b65b8-rw26k\" (UID: \"16652170-c20c-4344-9e2c-55d0f861edc2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rw26k"
Sep 07 09:05:08.455073 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:08.455016    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwtb9\" (UniqueName: \"kubernetes.io/projected/16652170-c20c-4344-9e2c-55d0f861edc2-kube-api-access-rwtb9\") pod \"kserve-controller-manager-d6f4b65b8-rw26k\" (UID: \"16652170-c20c-4344-9e2c-55d0f861edc2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rw26k"
Sep 07 09:05:08.476742 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:08.476698    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:08.556450 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:08.556426    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwtb9\" (UniqueName: \"kubernetes.io/projected/16652170-c20c-4344-9e2c-55d0f861edc2-kube-api-access-rwtb9\") pod \"kserve-controller-manager-d6f4b65b8-rw26k\" (UID: \"16652170-c20c-4344-9e2c-55d0f861edc2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rw26k"
Sep 07 09:05:08.556575 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:08.556518    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16652170-c20c-4344-9e2c-55d0f861edc2-cert\") pod \"kserve-controller-manager-d6f4b65b8-rw26k\" (UID: \"16652170-c20c-4344-9e2c-55d0f861edc2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rw26k"
Sep 07 09:05:08.556637 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:05:08.556624    2574 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 07 09:05:08.556706 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:05:08.556696    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16652170-c20c-4344-9e2c-55d0f861edc2-cert podName:16652170-c20c-4344-9e2c-55d0f861edc2 nodeName:}" failed. No retries permitted until 2026-09-07 09:05:09.056676018 +0000 UTC m=+392.020592221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/16652170-c20c-4344-9e2c-55d0f861edc2-cert") pod "kserve-controller-manager-d6f4b65b8-rw26k" (UID: "16652170-c20c-4344-9e2c-55d0f861edc2") : secret "kserve-webhook-server-cert" not found
Sep 07 09:05:08.577910 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:08.574924    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwtb9\" (UniqueName: \"kubernetes.io/projected/16652170-c20c-4344-9e2c-55d0f861edc2-kube-api-access-rwtb9\") pod \"kserve-controller-manager-d6f4b65b8-rw26k\" (UID: \"16652170-c20c-4344-9e2c-55d0f861edc2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rw26k"
Sep 07 09:05:08.668545 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:08.668515    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:08.737278 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:08.737245    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:09.061106 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:09.061020    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16652170-c20c-4344-9e2c-55d0f861edc2-cert\") pod \"kserve-controller-manager-d6f4b65b8-rw26k\" (UID: \"16652170-c20c-4344-9e2c-55d0f861edc2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rw26k"
Sep 07 09:05:09.063587 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:09.063557    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/16652170-c20c-4344-9e2c-55d0f861edc2-cert\") pod \"kserve-controller-manager-d6f4b65b8-rw26k\" (UID: \"16652170-c20c-4344-9e2c-55d0f861edc2\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rw26k"
Sep 07 09:05:09.319270 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:09.319199    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rw26k"
Sep 07 09:05:09.598741 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:09.596214    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rw26k"]
Sep 07 09:05:09.601957 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:05:09.601930    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16652170_c20c_4344_9e2c_55d0f861edc2.slice/crio-947e0ae81e46788781f4c2c0eafe08ac4acafb41d1b1af598c41feb4e536e398 WatchSource:0}: Error finding container 947e0ae81e46788781f4c2c0eafe08ac4acafb41d1b1af598c41feb4e536e398: Status 404 returned error can't find the container with id 947e0ae81e46788781f4c2c0eafe08ac4acafb41d1b1af598c41feb4e536e398
Sep 07 09:05:09.679383 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:09.679348    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:09.779070 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:09.779034    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:10.370299 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.370261    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rw26k" event={"ID":"16652170-c20c-4344-9e2c-55d0f861edc2","Type":"ContainerStarted","Data":"947e0ae81e46788781f4c2c0eafe08ac4acafb41d1b1af598c41feb4e536e398"}
Sep 07 09:05:10.683007 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.682964    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"]
Sep 07 09:05:10.683449 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.683188    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc" podUID="223c8f38-0d96-4145-aec4-57b8c4148e57" containerName="discovery" containerID="cri-o://8b99413031d57dd6697569836411edae0e330ee4949269fe9072998e3a5de0a4" gracePeriod=30
Sep 07 09:05:10.936836 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.936773    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:05:10.982203 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.982169    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-kubeconfig\") pod \"223c8f38-0d96-4145-aec4-57b8c4148e57\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") "
Sep 07 09:05:10.982334 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.982258    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-token\") pod \"223c8f38-0d96-4145-aec4-57b8c4148e57\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") "
Sep 07 09:05:10.982334 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.982293    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-csr-ca-configmap\") pod \"223c8f38-0d96-4145-aec4-57b8c4148e57\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") "
Sep 07 09:05:10.982458 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.982339    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-csr-dns-cert\") pod \"223c8f38-0d96-4145-aec4-57b8c4148e57\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") "
Sep 07 09:05:10.982458 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.982365    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fn9lv\" (UniqueName: \"kubernetes.io/projected/223c8f38-0d96-4145-aec4-57b8c4148e57-kube-api-access-fn9lv\") pod \"223c8f38-0d96-4145-aec4-57b8c4148e57\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") "
Sep 07 09:05:10.982458 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.982394    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/223c8f38-0d96-4145-aec4-57b8c4148e57-local-certs\") pod \"223c8f38-0d96-4145-aec4-57b8c4148e57\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") "
Sep 07 09:05:10.982458 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.982428    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-cacerts\") pod \"223c8f38-0d96-4145-aec4-57b8c4148e57\" (UID: \"223c8f38-0d96-4145-aec4-57b8c4148e57\") "
Sep 07 09:05:10.982688 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.982659    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "223c8f38-0d96-4145-aec4-57b8c4148e57" (UID: "223c8f38-0d96-4145-aec4-57b8c4148e57"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:05:10.984930 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.984899    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223c8f38-0d96-4145-aec4-57b8c4148e57-local-certs" (OuterVolumeSpecName: "local-certs") pod "223c8f38-0d96-4145-aec4-57b8c4148e57" (UID: "223c8f38-0d96-4145-aec4-57b8c4148e57"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:05:10.985090 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.985068    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "223c8f38-0d96-4145-aec4-57b8c4148e57" (UID: "223c8f38-0d96-4145-aec4-57b8c4148e57"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:05:10.985186 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.985108    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-token" (OuterVolumeSpecName: "istio-token") pod "223c8f38-0d96-4145-aec4-57b8c4148e57" (UID: "223c8f38-0d96-4145-aec4-57b8c4148e57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:05:10.985250 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.985199    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-cacerts" (OuterVolumeSpecName: "cacerts") pod "223c8f38-0d96-4145-aec4-57b8c4148e57" (UID: "223c8f38-0d96-4145-aec4-57b8c4148e57"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:05:10.985606 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.985581    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "223c8f38-0d96-4145-aec4-57b8c4148e57" (UID: "223c8f38-0d96-4145-aec4-57b8c4148e57"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:05:10.985606 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:10.985589    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/223c8f38-0d96-4145-aec4-57b8c4148e57-kube-api-access-fn9lv" (OuterVolumeSpecName: "kube-api-access-fn9lv") pod "223c8f38-0d96-4145-aec4-57b8c4148e57" (UID: "223c8f38-0d96-4145-aec4-57b8c4148e57"). InnerVolumeSpecName "kube-api-access-fn9lv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:05:11.083053 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.083012    2574 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-kubeconfig\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:05:11.083053 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.083049    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-token\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:05:11.083053 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.083058    2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-csr-ca-configmap\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:05:11.083053 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.083067    2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-istio-csr-dns-cert\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:05:11.083304 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.083076    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn9lv\" (UniqueName: \"kubernetes.io/projected/223c8f38-0d96-4145-aec4-57b8c4148e57-kube-api-access-fn9lv\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:05:11.083304 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.083083    2574 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/223c8f38-0d96-4145-aec4-57b8c4148e57-local-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:05:11.083304 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.083091    2574 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/223c8f38-0d96-4145-aec4-57b8c4148e57-cacerts\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:05:11.376227 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.376152    2574 generic.go:356] "Generic (PLEG): container finished" podID="223c8f38-0d96-4145-aec4-57b8c4148e57" containerID="8b99413031d57dd6697569836411edae0e330ee4949269fe9072998e3a5de0a4" exitCode=0
Sep 07 09:05:11.376227 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.376214    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"
Sep 07 09:05:11.376425 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.376236    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc" event={"ID":"223c8f38-0d96-4145-aec4-57b8c4148e57","Type":"ContainerDied","Data":"8b99413031d57dd6697569836411edae0e330ee4949269fe9072998e3a5de0a4"}
Sep 07 09:05:11.376425 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.376300    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc" event={"ID":"223c8f38-0d96-4145-aec4-57b8c4148e57","Type":"ContainerDied","Data":"33308260930b35ef92a39d82fe514591d1f6041e351a62600464ea64445fbfd6"}
Sep 07 09:05:11.376425 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.376329    2574 scope.go:117] "RemoveContainer" containerID="8b99413031d57dd6697569836411edae0e330ee4949269fe9072998e3a5de0a4"
Sep 07 09:05:11.405386 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.405356    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"]
Sep 07 09:05:11.409956 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.409937    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bnpdc"]
Sep 07 09:05:11.543620 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.543585    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="223c8f38-0d96-4145-aec4-57b8c4148e57" path="/var/lib/kubelet/pods/223c8f38-0d96-4145-aec4-57b8c4148e57/volumes"
Sep 07 09:05:11.862517 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.862346    2574 scope.go:117] "RemoveContainer" containerID="8b99413031d57dd6697569836411edae0e330ee4949269fe9072998e3a5de0a4"
Sep 07 09:05:11.862763 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:05:11.862726    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b99413031d57dd6697569836411edae0e330ee4949269fe9072998e3a5de0a4\": container with ID starting with 8b99413031d57dd6697569836411edae0e330ee4949269fe9072998e3a5de0a4 not found: ID does not exist" containerID="8b99413031d57dd6697569836411edae0e330ee4949269fe9072998e3a5de0a4"
Sep 07 09:05:11.862834 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:11.862755    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b99413031d57dd6697569836411edae0e330ee4949269fe9072998e3a5de0a4"} err="failed to get container status \"8b99413031d57dd6697569836411edae0e330ee4949269fe9072998e3a5de0a4\": rpc error: code = NotFound desc = could not find container \"8b99413031d57dd6697569836411edae0e330ee4949269fe9072998e3a5de0a4\": container with ID starting with 8b99413031d57dd6697569836411edae0e330ee4949269fe9072998e3a5de0a4 not found: ID does not exist"
Sep 07 09:05:12.382847 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:12.382818    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rw26k" event={"ID":"16652170-c20c-4344-9e2c-55d0f861edc2","Type":"ContainerStarted","Data":"ba6d29913d65053cd36b2af87d22e18e8f87eaeaa13dce18e80244e60748778c"}
Sep 07 09:05:12.383025 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:12.382949    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rw26k"
Sep 07 09:05:12.397807 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:12.397765    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rw26k" podStartSLOduration=2.092678602 podStartE2EDuration="4.397753148s" totalImagesPullingTime="2.305074546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:05:12.396400373 +0000 UTC m=+395.360316608" watchObservedRunningTime="2026-09-07 09:05:12.397753148 +0000 UTC m=+395.361669368"
Sep 07 09:05:39.973616 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:39.973580    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:40.063133 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:40.063102    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:40.564400 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:40.564363    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:40.665081 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:40.665048    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:43.393625 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:43.393595    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rw26k"
Sep 07 09:05:50.971406 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:50.971371    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:51.026533 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:51.026491    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:05:51.084544 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:05:51.084510    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:00.547571 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:00.547467    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:00.647389 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:00.647354    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:15.619909 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:15.619879    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:16.653090 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:16.653047    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:19.231924 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:19.231859    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:20.254530 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:20.254496    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-qdsqv"]
Sep 07 09:06:20.254898 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:20.254805    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="223c8f38-0d96-4145-aec4-57b8c4148e57" containerName="discovery"
Sep 07 09:06:20.254898 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:20.254816    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="223c8f38-0d96-4145-aec4-57b8c4148e57" containerName="discovery"
Sep 07 09:06:20.254898 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:20.254870    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="223c8f38-0d96-4145-aec4-57b8c4148e57" containerName="discovery"
Sep 07 09:06:20.256752 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:20.256737    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-qdsqv"
Sep 07 09:06:20.258683 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:20.258659    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 07 09:06:20.258683 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:20.258673    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 07 09:06:20.258890 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:20.258742    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-llbl4\""
Sep 07 09:06:20.268484 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:20.268466    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-qdsqv"]
Sep 07 09:06:20.326686 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:20.326665    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m4hv\" (UniqueName: \"kubernetes.io/projected/1250b781-ccde-4c52-89e6-fa72b8349f33-kube-api-access-5m4hv\") pod \"authorino-operator-68d88948d8-qdsqv\" (UID: \"1250b781-ccde-4c52-89e6-fa72b8349f33\") " pod="kuadrant-system/authorino-operator-68d88948d8-qdsqv"
Sep 07 09:06:20.427619 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:20.427596    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m4hv\" (UniqueName: \"kubernetes.io/projected/1250b781-ccde-4c52-89e6-fa72b8349f33-kube-api-access-5m4hv\") pod \"authorino-operator-68d88948d8-qdsqv\" (UID: \"1250b781-ccde-4c52-89e6-fa72b8349f33\") " pod="kuadrant-system/authorino-operator-68d88948d8-qdsqv"
Sep 07 09:06:20.437593 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:20.437568    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m4hv\" (UniqueName: \"kubernetes.io/projected/1250b781-ccde-4c52-89e6-fa72b8349f33-kube-api-access-5m4hv\") pod \"authorino-operator-68d88948d8-qdsqv\" (UID: \"1250b781-ccde-4c52-89e6-fa72b8349f33\") " pod="kuadrant-system/authorino-operator-68d88948d8-qdsqv"
Sep 07 09:06:20.566124 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:20.566064    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-qdsqv"
Sep 07 09:06:20.856516 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:20.856490    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-qdsqv"]
Sep 07 09:06:20.858861 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:06:20.858839    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1250b781_ccde_4c52_89e6_fa72b8349f33.slice/crio-2dc05fc7e15f9a5b6b22c7a0d73a57887081da56f5dfbdd5cd7ab8e95b997119 WatchSource:0}: Error finding container 2dc05fc7e15f9a5b6b22c7a0d73a57887081da56f5dfbdd5cd7ab8e95b997119: Status 404 returned error can't find the container with id 2dc05fc7e15f9a5b6b22c7a0d73a57887081da56f5dfbdd5cd7ab8e95b997119
Sep 07 09:06:21.386587 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:21.386554    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:21.675966 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:21.675934    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-qdsqv" event={"ID":"1250b781-ccde-4c52-89e6-fa72b8349f33","Type":"ContainerStarted","Data":"2dc05fc7e15f9a5b6b22c7a0d73a57887081da56f5dfbdd5cd7ab8e95b997119"}
Sep 07 09:06:22.027783 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:22.027670    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:22.262811 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:22.262774    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:22.373035 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:22.372959    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:23.696926 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:23.696896    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"]
Sep 07 09:06:23.700503 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:23.700486    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"
Sep 07 09:06:23.702454 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:23.702437    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-7hqrv\""
Sep 07 09:06:23.712958 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:23.712936    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"]
Sep 07 09:06:23.754178 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:23.754154    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knnn7\" (UniqueName: \"kubernetes.io/projected/7583a238-c434-457d-aa6d-989bdcb67230-kube-api-access-knnn7\") pod \"limitador-operator-controller-manager-6c549b6d85-ls9z6\" (UID: \"7583a238-c434-457d-aa6d-989bdcb67230\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"
Sep 07 09:06:23.855313 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:23.855286    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knnn7\" (UniqueName: \"kubernetes.io/projected/7583a238-c434-457d-aa6d-989bdcb67230-kube-api-access-knnn7\") pod \"limitador-operator-controller-manager-6c549b6d85-ls9z6\" (UID: \"7583a238-c434-457d-aa6d-989bdcb67230\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"
Sep 07 09:06:23.865263 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:23.865235    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knnn7\" (UniqueName: \"kubernetes.io/projected/7583a238-c434-457d-aa6d-989bdcb67230-kube-api-access-knnn7\") pod \"limitador-operator-controller-manager-6c549b6d85-ls9z6\" (UID: \"7583a238-c434-457d-aa6d-989bdcb67230\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"
Sep 07 09:06:24.010694 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:24.010637    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"
Sep 07 09:06:24.224853 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:24.224815    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:24.285003 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:24.284982    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"]
Sep 07 09:06:24.322142 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:24.322113    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:24.690223 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:24.690195    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6" event={"ID":"7583a238-c434-457d-aa6d-989bdcb67230","Type":"ContainerStarted","Data":"3595ef065b9c0dcd1bf57bdc0931838b04774b093f4eac9fadea0898194d558e"}
Sep 07 09:06:24.809392 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:24.809356    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:25.418205 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:25.418145    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:27.204116 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:27.204085    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-dkdh8"]
Sep 07 09:06:27.206547 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:27.206526    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-dkdh8"
Sep 07 09:06:27.208901 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:27.208876    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 07 09:06:27.209223 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:27.209205    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-9f27p\""
Sep 07 09:06:27.220028 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:27.220009    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-dkdh8"]
Sep 07 09:06:27.285005 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:27.284980    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw7t2\" (UniqueName: \"kubernetes.io/projected/e831e301-5424-4ef1-aa22-145cce2b0656-kube-api-access-tw7t2\") pod \"dns-operator-controller-manager-5985564776-dkdh8\" (UID: \"e831e301-5424-4ef1-aa22-145cce2b0656\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-dkdh8"
Sep 07 09:06:27.386307 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:27.386275    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw7t2\" (UniqueName: \"kubernetes.io/projected/e831e301-5424-4ef1-aa22-145cce2b0656-kube-api-access-tw7t2\") pod \"dns-operator-controller-manager-5985564776-dkdh8\" (UID: \"e831e301-5424-4ef1-aa22-145cce2b0656\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-dkdh8"
Sep 07 09:06:27.395341 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:27.395317    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw7t2\" (UniqueName: \"kubernetes.io/projected/e831e301-5424-4ef1-aa22-145cce2b0656-kube-api-access-tw7t2\") pod \"dns-operator-controller-manager-5985564776-dkdh8\" (UID: \"e831e301-5424-4ef1-aa22-145cce2b0656\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-dkdh8"
Sep 07 09:06:27.517557 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:27.517462    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-dkdh8"
Sep 07 09:06:27.705752 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:27.705697    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6" event={"ID":"7583a238-c434-457d-aa6d-989bdcb67230","Type":"ContainerStarted","Data":"b2c53db445739f52826ac5520a34b51023a6f64d4961761b641c2d48964ac076"}
Sep 07 09:06:27.705917 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:27.705855    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"
Sep 07 09:06:27.740982 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:27.740939    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6" podStartSLOduration=2.318847715 podStartE2EDuration="4.740924019s" totalImagesPullingTime="2.422076304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:06:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:06:27.739494259 +0000 UTC m=+470.703410481" watchObservedRunningTime="2026-09-07 09:06:27.740924019 +0000 UTC m=+470.704840240"
Sep 07 09:06:27.792446 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:27.792367    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-dkdh8"]
Sep 07 09:06:27.794647 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:06:27.794623    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode831e301_5424_4ef1_aa22_145cce2b0656.slice/crio-d57b8e33a6a77016514b1de8407604d8044afc831e28f9fb0384afed783ee51f WatchSource:0}: Error finding container d57b8e33a6a77016514b1de8407604d8044afc831e28f9fb0384afed783ee51f: Status 404 returned error can't find the container with id d57b8e33a6a77016514b1de8407604d8044afc831e28f9fb0384afed783ee51f
Sep 07 09:06:27.822826 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:27.822797    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:28.595228 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:28.595196    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:28.715448 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:28.715409    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-dkdh8" event={"ID":"e831e301-5424-4ef1-aa22-145cce2b0656","Type":"ContainerStarted","Data":"d57b8e33a6a77016514b1de8407604d8044afc831e28f9fb0384afed783ee51f"}
Sep 07 09:06:29.187973 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:29.187938    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:29.529211 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:29.529136    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:29.721463 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:29.721434    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-dkdh8" event={"ID":"e831e301-5424-4ef1-aa22-145cce2b0656","Type":"ContainerStarted","Data":"e966831b1bec08f84745dc9ba4eed11793da1098e8b676271bf8f22f6c3135da"}
Sep 07 09:06:29.721843 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:29.721504    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-dkdh8"
Sep 07 09:06:29.738575 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:29.738525    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-dkdh8" podStartSLOduration=1.230922834 podStartE2EDuration="2.738509726s" totalImagesPullingTime="1.507586892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:06:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:06:29.737428927 +0000 UTC m=+472.701345167" watchObservedRunningTime="2026-09-07 09:06:29.738509726 +0000 UTC m=+472.702425947"
Sep 07 09:06:31.481113 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:31.481077    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:31.593049 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:31.593018    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:33.437848 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:33.437813    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"]
Sep 07 09:06:33.440125 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:33.440109    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:33.442097 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:33.442072    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-s4rqb\""
Sep 07 09:06:33.455245 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:33.455224    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"]
Sep 07 09:06:33.536070 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:33.536043    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r6gn\" (UniqueName: \"kubernetes.io/projected/ca392c92-fbf2-4fd0-bc8b-e0a57331e8af-kube-api-access-4r6gn\") pod \"kuadrant-operator-controller-manager-58fc56757c-spngz\" (UID: \"ca392c92-fbf2-4fd0-bc8b-e0a57331e8af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:33.536191 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:33.536079    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca392c92-fbf2-4fd0-bc8b-e0a57331e8af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-spngz\" (UID: \"ca392c92-fbf2-4fd0-bc8b-e0a57331e8af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:33.637247 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:33.637213    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r6gn\" (UniqueName: \"kubernetes.io/projected/ca392c92-fbf2-4fd0-bc8b-e0a57331e8af-kube-api-access-4r6gn\") pod \"kuadrant-operator-controller-manager-58fc56757c-spngz\" (UID: \"ca392c92-fbf2-4fd0-bc8b-e0a57331e8af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:33.637407 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:33.637263    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca392c92-fbf2-4fd0-bc8b-e0a57331e8af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-spngz\" (UID: \"ca392c92-fbf2-4fd0-bc8b-e0a57331e8af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:33.642740 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:33.638229    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca392c92-fbf2-4fd0-bc8b-e0a57331e8af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-spngz\" (UID: \"ca392c92-fbf2-4fd0-bc8b-e0a57331e8af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:33.645871 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:33.645845    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r6gn\" (UniqueName: \"kubernetes.io/projected/ca392c92-fbf2-4fd0-bc8b-e0a57331e8af-kube-api-access-4r6gn\") pod \"kuadrant-operator-controller-manager-58fc56757c-spngz\" (UID: \"ca392c92-fbf2-4fd0-bc8b-e0a57331e8af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:33.751081 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:33.751019    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:34.033969 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:34.033944    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"]
Sep 07 09:06:34.036147 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:06:34.036125    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca392c92_fbf2_4fd0_bc8b_e0a57331e8af.slice/crio-7fc0673158b8946ce4a4bbe8f148f9f0e20d5443cfc3524680981237d22a55d1 WatchSource:0}: Error finding container 7fc0673158b8946ce4a4bbe8f148f9f0e20d5443cfc3524680981237d22a55d1: Status 404 returned error can't find the container with id 7fc0673158b8946ce4a4bbe8f148f9f0e20d5443cfc3524680981237d22a55d1
Sep 07 09:06:34.743569 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:34.743535    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz" event={"ID":"ca392c92-fbf2-4fd0-bc8b-e0a57331e8af","Type":"ContainerStarted","Data":"7fc0673158b8946ce4a4bbe8f148f9f0e20d5443cfc3524680981237d22a55d1"}
Sep 07 09:06:35.586468 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:35.586439    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:35.689261 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:35.689228    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:36.788156 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:36.788123    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:36.889762 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:36.889727    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:38.718656 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:38.718626    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"
Sep 07 09:06:39.771858 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:39.771827    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-qdsqv" event={"ID":"1250b781-ccde-4c52-89e6-fa72b8349f33","Type":"ContainerStarted","Data":"2a400d5ae456487239781f6d6a31a82ca6ab0c0393f30153b535989f17927047"}
Sep 07 09:06:39.772254 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:39.771940    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-qdsqv"
Sep 07 09:06:39.773674 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:39.773652    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz" event={"ID":"ca392c92-fbf2-4fd0-bc8b-e0a57331e8af","Type":"ContainerStarted","Data":"6c61e06e05c00f85638012c37c7ad0d489780031d03123ba65d54c348e72fe55"}
Sep 07 09:06:39.773808 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:39.773756    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:39.788246 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:39.788207    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-qdsqv" podStartSLOduration=1.698862041 podStartE2EDuration="19.788194809s" totalImagesPullingTime="18.089332768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:06:39.78689888 +0000 UTC m=+482.750815128" watchObservedRunningTime="2026-09-07 09:06:39.788194809 +0000 UTC m=+482.752111030"
Sep 07 09:06:39.804911 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:39.804873    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz" podStartSLOduration=1.8724575959999998 podStartE2EDuration="6.804861766s" totalImagesPullingTime="4.93240417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:06:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:06:39.802909635 +0000 UTC m=+482.766825855" watchObservedRunningTime="2026-09-07 09:06:39.804861766 +0000 UTC m=+482.768777986"
Sep 07 09:06:40.727822 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:40.727790    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-dkdh8"
Sep 07 09:06:41.407500 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.407465    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:41.514912 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.514881    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:41.542041 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.542012    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz"]
Sep 07 09:06:41.544934 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.544913    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz"
Sep 07 09:06:41.547015 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.546987    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 07 09:06:41.547015 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.547005    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5x592\""
Sep 07 09:06:41.547200 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.547005    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 07 09:06:41.554984 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.554964    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz"]
Sep 07 09:06:41.607853 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.607824    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:41.708442 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.708332    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:41.709285 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.709267    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjvn9\" (UniqueName: \"kubernetes.io/projected/aecaad15-d260-416a-ba26-c59365d9cd5d-kube-api-access-gjvn9\") pod \"kuadrant-console-plugin-598f86468b-r9tlz\" (UID: \"aecaad15-d260-416a-ba26-c59365d9cd5d\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz"
Sep 07 09:06:41.709394 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.709300    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/aecaad15-d260-416a-ba26-c59365d9cd5d-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-r9tlz\" (UID: \"aecaad15-d260-416a-ba26-c59365d9cd5d\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz"
Sep 07 09:06:41.709394 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.709350    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/aecaad15-d260-416a-ba26-c59365d9cd5d-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-r9tlz\" (UID: \"aecaad15-d260-416a-ba26-c59365d9cd5d\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz"
Sep 07 09:06:41.810038 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.810014    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjvn9\" (UniqueName: \"kubernetes.io/projected/aecaad15-d260-416a-ba26-c59365d9cd5d-kube-api-access-gjvn9\") pod \"kuadrant-console-plugin-598f86468b-r9tlz\" (UID: \"aecaad15-d260-416a-ba26-c59365d9cd5d\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz"
Sep 07 09:06:41.810183 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.810047    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/aecaad15-d260-416a-ba26-c59365d9cd5d-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-r9tlz\" (UID: \"aecaad15-d260-416a-ba26-c59365d9cd5d\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz"
Sep 07 09:06:41.810183 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.810077    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/aecaad15-d260-416a-ba26-c59365d9cd5d-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-r9tlz\" (UID: \"aecaad15-d260-416a-ba26-c59365d9cd5d\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz"
Sep 07 09:06:41.810866 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.810832    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/aecaad15-d260-416a-ba26-c59365d9cd5d-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-r9tlz\" (UID: \"aecaad15-d260-416a-ba26-c59365d9cd5d\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz"
Sep 07 09:06:41.812642 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.812616    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/aecaad15-d260-416a-ba26-c59365d9cd5d-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-r9tlz\" (UID: \"aecaad15-d260-416a-ba26-c59365d9cd5d\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz"
Sep 07 09:06:41.816968 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.816945    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjvn9\" (UniqueName: \"kubernetes.io/projected/aecaad15-d260-416a-ba26-c59365d9cd5d-kube-api-access-gjvn9\") pod \"kuadrant-console-plugin-598f86468b-r9tlz\" (UID: \"aecaad15-d260-416a-ba26-c59365d9cd5d\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz"
Sep 07 09:06:41.856390 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:41.856372    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz"
Sep 07 09:06:42.123684 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:42.123619    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz"]
Sep 07 09:06:42.125943 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:06:42.125921    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaecaad15_d260_416a_ba26_c59365d9cd5d.slice/crio-433c0cc0e740fb78751466e689f8b21f17dfb9bfa0b534e13720b80f393ffd35 WatchSource:0}: Error finding container 433c0cc0e740fb78751466e689f8b21f17dfb9bfa0b534e13720b80f393ffd35: Status 404 returned error can't find the container with id 433c0cc0e740fb78751466e689f8b21f17dfb9bfa0b534e13720b80f393ffd35
Sep 07 09:06:42.788699 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:42.788664    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz" event={"ID":"aecaad15-d260-416a-ba26-c59365d9cd5d","Type":"ContainerStarted","Data":"433c0cc0e740fb78751466e689f8b21f17dfb9bfa0b534e13720b80f393ffd35"}
Sep 07 09:06:45.389255 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:45.389218    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:45.442876 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:45.442838    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:47.818418 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:47.818383    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz" event={"ID":"aecaad15-d260-416a-ba26-c59365d9cd5d","Type":"ContainerStarted","Data":"65d365d95d4ef21d41fb282d6625fd2a74aa0b88ca9a844940a70a1b9fd4e9b2"}
Sep 07 09:06:47.832730 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:47.832666    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-r9tlz" podStartSLOduration=1.839320232 podStartE2EDuration="6.832652982s" totalImagesPullingTime="4.99333275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:06:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:06:47.831559444 +0000 UTC m=+490.795475665" watchObservedRunningTime="2026-09-07 09:06:47.832652982 +0000 UTC m=+490.796569202"
Sep 07 09:06:50.781210 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:50.781176    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:50.781648 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:50.781323    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-qdsqv"
Sep 07 09:06:51.526008 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:51.525970    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:51.651726 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:51.651678    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:51.849084 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:51.848989    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:51.949632 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:51.949594    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:52.203927 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:52.203864    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:52.307727 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:52.307686    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:52.938788 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:52.938752    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"]
Sep 07 09:06:52.939157 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:52.938952    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz" podUID="ca392c92-fbf2-4fd0-bc8b-e0a57331e8af" containerName="manager" containerID="cri-o://6c61e06e05c00f85638012c37c7ad0d489780031d03123ba65d54c348e72fe55" gracePeriod=2
Sep 07 09:06:52.943506 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:52.943481    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"]
Sep 07 09:06:52.970495 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:52.970471    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"]
Sep 07 09:06:52.970957 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:52.970937    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca392c92-fbf2-4fd0-bc8b-e0a57331e8af" containerName="manager"
Sep 07 09:06:52.970957 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:52.970958    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca392c92-fbf2-4fd0-bc8b-e0a57331e8af" containerName="manager"
Sep 07 09:06:52.971116 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:52.971099    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca392c92-fbf2-4fd0-bc8b-e0a57331e8af" containerName="manager"
Sep 07 09:06:52.973249 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:52.973227    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"
Sep 07 09:06:52.975122 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:06:52.975094    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-spngz\" is forbidden: User \"system:node:ip-10-0-141-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-81.ec2.internal' and this object" podUID="ca392c92-fbf2-4fd0-bc8b-e0a57331e8af" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:52.980115 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:52.980094    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"]
Sep 07 09:06:52.980261 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:52.980244    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6" podUID="7583a238-c434-457d-aa6d-989bdcb67230" containerName="manager" containerID="cri-o://b2c53db445739f52826ac5520a34b51023a6f64d4961761b641c2d48964ac076" gracePeriod=2
Sep 07 09:06:52.989684 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:52.989650    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"]
Sep 07 09:06:52.992043 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:52.992021    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"]
Sep 07 09:06:53.003739 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:06:53.003695    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-ls9z6\" is forbidden: User \"system:node:ip-10-0-141-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-81.ec2.internal' and this object" podUID="7583a238-c434-457d-aa6d-989bdcb67230" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"
Sep 07 09:06:53.004084 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.004063    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9877p\" (UniqueName: \"kubernetes.io/projected/bb706726-28a3-4424-84d8-26636b1b5e03-kube-api-access-9877p\") pod \"kuadrant-operator-controller-manager-58fc56757c-db8cx\" (UID: \"bb706726-28a3-4424-84d8-26636b1b5e03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"
Sep 07 09:06:53.004156 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.004122    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb706726-28a3-4424-84d8-26636b1b5e03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-db8cx\" (UID: \"bb706726-28a3-4424-84d8-26636b1b5e03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"
Sep 07 09:06:53.009075 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:06:53.009034    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-spngz\" is forbidden: User \"system:node:ip-10-0-141-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-81.ec2.internal' and this object" podUID="ca392c92-fbf2-4fd0-bc8b-e0a57331e8af" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:53.011328 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.011304    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fbsf7"]
Sep 07 09:06:53.011856 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.011835    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7583a238-c434-457d-aa6d-989bdcb67230" containerName="manager"
Sep 07 09:06:53.011907 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.011858    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7583a238-c434-457d-aa6d-989bdcb67230" containerName="manager"
Sep 07 09:06:53.011981 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.011969    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7583a238-c434-457d-aa6d-989bdcb67230" containerName="manager"
Sep 07 09:06:53.014348 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.014329    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fbsf7"
Sep 07 09:06:53.023756 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.023691    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fbsf7"]
Sep 07 09:06:53.037826 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:06:53.037795    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-ls9z6\" is forbidden: User \"system:node:ip-10-0-141-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-81.ec2.internal' and this object" podUID="7583a238-c434-457d-aa6d-989bdcb67230" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"
Sep 07 09:06:53.039497 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:06:53.039464    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-spngz\" is forbidden: User \"system:node:ip-10-0-141-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-81.ec2.internal' and this object" podUID="ca392c92-fbf2-4fd0-bc8b-e0a57331e8af" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:53.105496 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.105472    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9877p\" (UniqueName: \"kubernetes.io/projected/bb706726-28a3-4424-84d8-26636b1b5e03-kube-api-access-9877p\") pod \"kuadrant-operator-controller-manager-58fc56757c-db8cx\" (UID: \"bb706726-28a3-4424-84d8-26636b1b5e03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"
Sep 07 09:06:53.105628 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.105517    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb706726-28a3-4424-84d8-26636b1b5e03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-db8cx\" (UID: \"bb706726-28a3-4424-84d8-26636b1b5e03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"
Sep 07 09:06:53.105628 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.105588    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsfmj\" (UniqueName: \"kubernetes.io/projected/908a409f-da1f-4089-88b0-8c9651eb97a9-kube-api-access-hsfmj\") pod \"limitador-operator-controller-manager-6c549b6d85-fbsf7\" (UID: \"908a409f-da1f-4089-88b0-8c9651eb97a9\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fbsf7"
Sep 07 09:06:53.106021 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.105991    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb706726-28a3-4424-84d8-26636b1b5e03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-db8cx\" (UID: \"bb706726-28a3-4424-84d8-26636b1b5e03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"
Sep 07 09:06:53.114788 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.113790    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9877p\" (UniqueName: \"kubernetes.io/projected/bb706726-28a3-4424-84d8-26636b1b5e03-kube-api-access-9877p\") pod \"kuadrant-operator-controller-manager-58fc56757c-db8cx\" (UID: \"bb706726-28a3-4424-84d8-26636b1b5e03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"
Sep 07 09:06:53.206299 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.206268    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsfmj\" (UniqueName: \"kubernetes.io/projected/908a409f-da1f-4089-88b0-8c9651eb97a9-kube-api-access-hsfmj\") pod \"limitador-operator-controller-manager-6c549b6d85-fbsf7\" (UID: \"908a409f-da1f-4089-88b0-8c9651eb97a9\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fbsf7"
Sep 07 09:06:53.214910 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.214887    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsfmj\" (UniqueName: \"kubernetes.io/projected/908a409f-da1f-4089-88b0-8c9651eb97a9-kube-api-access-hsfmj\") pod \"limitador-operator-controller-manager-6c549b6d85-fbsf7\" (UID: \"908a409f-da1f-4089-88b0-8c9651eb97a9\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fbsf7"
Sep 07 09:06:53.221895 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.221878    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:53.223503 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:06:53.223480    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-ls9z6\" is forbidden: User \"system:node:ip-10-0-141-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-81.ec2.internal' and this object" podUID="7583a238-c434-457d-aa6d-989bdcb67230" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"
Sep 07 09:06:53.223791 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.223777    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"
Sep 07 09:06:53.224842 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:06:53.224824    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-spngz\" is forbidden: User \"system:node:ip-10-0-141-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-81.ec2.internal' and this object" podUID="ca392c92-fbf2-4fd0-bc8b-e0a57331e8af" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:53.226142 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:06:53.226117    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-ls9z6\" is forbidden: User \"system:node:ip-10-0-141-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-81.ec2.internal' and this object" podUID="7583a238-c434-457d-aa6d-989bdcb67230" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"
Sep 07 09:06:53.227440 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:06:53.227416    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-spngz\" is forbidden: User \"system:node:ip-10-0-141-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-81.ec2.internal' and this object" podUID="ca392c92-fbf2-4fd0-bc8b-e0a57331e8af" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:53.306643 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.306615    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r6gn\" (UniqueName: \"kubernetes.io/projected/ca392c92-fbf2-4fd0-bc8b-e0a57331e8af-kube-api-access-4r6gn\") pod \"ca392c92-fbf2-4fd0-bc8b-e0a57331e8af\" (UID: \"ca392c92-fbf2-4fd0-bc8b-e0a57331e8af\") "
Sep 07 09:06:53.306817 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.306683    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knnn7\" (UniqueName: \"kubernetes.io/projected/7583a238-c434-457d-aa6d-989bdcb67230-kube-api-access-knnn7\") pod \"7583a238-c434-457d-aa6d-989bdcb67230\" (UID: \"7583a238-c434-457d-aa6d-989bdcb67230\") "
Sep 07 09:06:53.306817 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.306753    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca392c92-fbf2-4fd0-bc8b-e0a57331e8af-extensions-socket-volume\") pod \"ca392c92-fbf2-4fd0-bc8b-e0a57331e8af\" (UID: \"ca392c92-fbf2-4fd0-bc8b-e0a57331e8af\") "
Sep 07 09:06:53.307202 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.307171    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca392c92-fbf2-4fd0-bc8b-e0a57331e8af-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ca392c92-fbf2-4fd0-bc8b-e0a57331e8af" (UID: "ca392c92-fbf2-4fd0-bc8b-e0a57331e8af"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:06:53.309022 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.309001    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7583a238-c434-457d-aa6d-989bdcb67230-kube-api-access-knnn7" (OuterVolumeSpecName: "kube-api-access-knnn7") pod "7583a238-c434-457d-aa6d-989bdcb67230" (UID: "7583a238-c434-457d-aa6d-989bdcb67230"). InnerVolumeSpecName "kube-api-access-knnn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:06:53.309087 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.309017    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca392c92-fbf2-4fd0-bc8b-e0a57331e8af-kube-api-access-4r6gn" (OuterVolumeSpecName: "kube-api-access-4r6gn") pod "ca392c92-fbf2-4fd0-bc8b-e0a57331e8af" (UID: "ca392c92-fbf2-4fd0-bc8b-e0a57331e8af"). InnerVolumeSpecName "kube-api-access-4r6gn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:06:53.357205 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.357174    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"
Sep 07 09:06:53.363591 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.363567    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fbsf7"
Sep 07 09:06:53.407389 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.407281    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r6gn\" (UniqueName: \"kubernetes.io/projected/ca392c92-fbf2-4fd0-bc8b-e0a57331e8af-kube-api-access-4r6gn\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:06:53.407389 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.407308    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knnn7\" (UniqueName: \"kubernetes.io/projected/7583a238-c434-457d-aa6d-989bdcb67230-kube-api-access-knnn7\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:06:53.407389 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.407325    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca392c92-fbf2-4fd0-bc8b-e0a57331e8af-extensions-socket-volume\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:06:53.541445 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.541364    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7583a238-c434-457d-aa6d-989bdcb67230" path="/var/lib/kubelet/pods/7583a238-c434-457d-aa6d-989bdcb67230/volumes"
Sep 07 09:06:53.541745 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.541732    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca392c92-fbf2-4fd0-bc8b-e0a57331e8af" path="/var/lib/kubelet/pods/ca392c92-fbf2-4fd0-bc8b-e0a57331e8af/volumes"
Sep 07 09:06:53.652762 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.652738    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"]
Sep 07 09:06:53.654241 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.654215    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fbsf7"]
Sep 07 09:06:53.654361 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:06:53.654279    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod908a409f_da1f_4089_88b0_8c9651eb97a9.slice/crio-c3d9062fa937d32aefe17b7ab2dc04e5453d4c769c1a02502f5e57a600732f55 WatchSource:0}: Error finding container c3d9062fa937d32aefe17b7ab2dc04e5453d4c769c1a02502f5e57a600732f55: Status 404 returned error can't find the container with id c3d9062fa937d32aefe17b7ab2dc04e5453d4c769c1a02502f5e57a600732f55
Sep 07 09:06:53.654990 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:06:53.654974    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb706726_28a3_4424_84d8_26636b1b5e03.slice/crio-fb08876470b36bf82ccbc0a2edba3413a04bd367db51a2039bb42ae9e67f1540 WatchSource:0}: Error finding container fb08876470b36bf82ccbc0a2edba3413a04bd367db51a2039bb42ae9e67f1540: Status 404 returned error can't find the container with id fb08876470b36bf82ccbc0a2edba3413a04bd367db51a2039bb42ae9e67f1540
Sep 07 09:06:53.848253 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.848176    2574 generic.go:356] "Generic (PLEG): container finished" podID="7583a238-c434-457d-aa6d-989bdcb67230" containerID="b2c53db445739f52826ac5520a34b51023a6f64d4961761b641c2d48964ac076" exitCode=0
Sep 07 09:06:53.848253 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.848221    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ls9z6"
Sep 07 09:06:53.848475 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.848284    2574 scope.go:117] "RemoveContainer" containerID="b2c53db445739f52826ac5520a34b51023a6f64d4961761b641c2d48964ac076"
Sep 07 09:06:53.850555 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.850530    2574 generic.go:356] "Generic (PLEG): container finished" podID="ca392c92-fbf2-4fd0-bc8b-e0a57331e8af" containerID="6c61e06e05c00f85638012c37c7ad0d489780031d03123ba65d54c348e72fe55" exitCode=0
Sep 07 09:06:53.850693 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.850572    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-spngz"
Sep 07 09:06:53.852789 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.852764    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fbsf7" event={"ID":"908a409f-da1f-4089-88b0-8c9651eb97a9","Type":"ContainerStarted","Data":"aa85efeecbd279f1e770ea3bb65539915d6998fae5ff85d4676d8f2ece11c6ad"}
Sep 07 09:06:53.852936 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.852816    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fbsf7" event={"ID":"908a409f-da1f-4089-88b0-8c9651eb97a9","Type":"ContainerStarted","Data":"c3d9062fa937d32aefe17b7ab2dc04e5453d4c769c1a02502f5e57a600732f55"}
Sep 07 09:06:53.853008 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.852953    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fbsf7"
Sep 07 09:06:53.854736 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.854688    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx" event={"ID":"bb706726-28a3-4424-84d8-26636b1b5e03","Type":"ContainerStarted","Data":"69cd8c89c0dcef9aa6b997b651ae183d1afed407e0879980aeb54fc99bad7146"}
Sep 07 09:06:53.854823 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.854748    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx" event={"ID":"bb706726-28a3-4424-84d8-26636b1b5e03","Type":"ContainerStarted","Data":"fb08876470b36bf82ccbc0a2edba3413a04bd367db51a2039bb42ae9e67f1540"}
Sep 07 09:06:53.854882 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.854844    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"
Sep 07 09:06:53.858535 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.858513    2574 scope.go:117] "RemoveContainer" containerID="b2c53db445739f52826ac5520a34b51023a6f64d4961761b641c2d48964ac076"
Sep 07 09:06:53.858877 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:06:53.858848    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2c53db445739f52826ac5520a34b51023a6f64d4961761b641c2d48964ac076\": container with ID starting with b2c53db445739f52826ac5520a34b51023a6f64d4961761b641c2d48964ac076 not found: ID does not exist" containerID="b2c53db445739f52826ac5520a34b51023a6f64d4961761b641c2d48964ac076"
Sep 07 09:06:53.858940 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.858889    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2c53db445739f52826ac5520a34b51023a6f64d4961761b641c2d48964ac076"} err="failed to get container status \"b2c53db445739f52826ac5520a34b51023a6f64d4961761b641c2d48964ac076\": rpc error: code = NotFound desc = could not find container \"b2c53db445739f52826ac5520a34b51023a6f64d4961761b641c2d48964ac076\": container with ID starting with b2c53db445739f52826ac5520a34b51023a6f64d4961761b641c2d48964ac076 not found: ID does not exist"
Sep 07 09:06:53.858940 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.858913    2574 scope.go:117] "RemoveContainer" containerID="6c61e06e05c00f85638012c37c7ad0d489780031d03123ba65d54c348e72fe55"
Sep 07 09:06:53.867093 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.867064    2574 scope.go:117] "RemoveContainer" containerID="6c61e06e05c00f85638012c37c7ad0d489780031d03123ba65d54c348e72fe55"
Sep 07 09:06:53.867355 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:06:53.867338    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c61e06e05c00f85638012c37c7ad0d489780031d03123ba65d54c348e72fe55\": container with ID starting with 6c61e06e05c00f85638012c37c7ad0d489780031d03123ba65d54c348e72fe55 not found: ID does not exist" containerID="6c61e06e05c00f85638012c37c7ad0d489780031d03123ba65d54c348e72fe55"
Sep 07 09:06:53.867413 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.867364    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c61e06e05c00f85638012c37c7ad0d489780031d03123ba65d54c348e72fe55"} err="failed to get container status \"6c61e06e05c00f85638012c37c7ad0d489780031d03123ba65d54c348e72fe55\": rpc error: code = NotFound desc = could not find container \"6c61e06e05c00f85638012c37c7ad0d489780031d03123ba65d54c348e72fe55\": container with ID starting with 6c61e06e05c00f85638012c37c7ad0d489780031d03123ba65d54c348e72fe55 not found: ID does not exist"
Sep 07 09:06:53.870603 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.870566    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fbsf7" podStartSLOduration=1.8705529840000001 podStartE2EDuration="1.870552984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:06:53.86934358 +0000 UTC m=+496.833259800" watchObservedRunningTime="2026-09-07 09:06:53.870552984 +0000 UTC m=+496.834469207"
Sep 07 09:06:53.890793 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:53.890756    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx" podStartSLOduration=1.890743254 podStartE2EDuration="1.890743254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:06:53.889468138 +0000 UTC m=+496.853384385" watchObservedRunningTime="2026-09-07 09:06:53.890743254 +0000 UTC m=+496.854659474"
Sep 07 09:06:59.784347 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:59.784301    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:06:59.838319 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:06:59.838282    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:00.582029 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:00.582002    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:00.645064 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:00.645030    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:03.678384 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:03.678347    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:04.865315 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:04.865282    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-fbsf7"
Sep 07 09:07:04.865803 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:04.865348    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"
Sep 07 09:07:05.793244 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:05.793216    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:05.889981 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:05.889950    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:08.605939 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.605902    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"]
Sep 07 09:07:08.606325 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.606082    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx" podUID="bb706726-28a3-4424-84d8-26636b1b5e03" containerName="manager" containerID="cri-o://69cd8c89c0dcef9aa6b997b651ae183d1afed407e0879980aeb54fc99bad7146" gracePeriod=10
Sep 07 09:07:08.811445 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.811405    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"]
Sep 07 09:07:08.833097 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.833065    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"]
Sep 07 09:07:08.833241 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.833168    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"
Sep 07 09:07:08.904943 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.904921    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"
Sep 07 09:07:08.931437 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.931410    2574 generic.go:356] "Generic (PLEG): container finished" podID="bb706726-28a3-4424-84d8-26636b1b5e03" containerID="69cd8c89c0dcef9aa6b997b651ae183d1afed407e0879980aeb54fc99bad7146" exitCode=0
Sep 07 09:07:08.931565 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.931466    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"
Sep 07 09:07:08.931565 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.931502    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx" event={"ID":"bb706726-28a3-4424-84d8-26636b1b5e03","Type":"ContainerDied","Data":"69cd8c89c0dcef9aa6b997b651ae183d1afed407e0879980aeb54fc99bad7146"}
Sep 07 09:07:08.931565 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.931555    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx" event={"ID":"bb706726-28a3-4424-84d8-26636b1b5e03","Type":"ContainerDied","Data":"fb08876470b36bf82ccbc0a2edba3413a04bd367db51a2039bb42ae9e67f1540"}
Sep 07 09:07:08.931695 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.931582    2574 scope.go:117] "RemoveContainer" containerID="69cd8c89c0dcef9aa6b997b651ae183d1afed407e0879980aeb54fc99bad7146"
Sep 07 09:07:08.936248 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.936226    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb706726-28a3-4424-84d8-26636b1b5e03-extensions-socket-volume\") pod \"bb706726-28a3-4424-84d8-26636b1b5e03\" (UID: \"bb706726-28a3-4424-84d8-26636b1b5e03\") "
Sep 07 09:07:08.936359 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.936290    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9877p\" (UniqueName: \"kubernetes.io/projected/bb706726-28a3-4424-84d8-26636b1b5e03-kube-api-access-9877p\") pod \"bb706726-28a3-4424-84d8-26636b1b5e03\" (UID: \"bb706726-28a3-4424-84d8-26636b1b5e03\") "
Sep 07 09:07:08.936441 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.936419    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5xrv\" (UniqueName: \"kubernetes.io/projected/6c18adc6-4851-4992-b4ab-12c3ad378dfe-kube-api-access-v5xrv\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-8rrhb\" (UID: \"6c18adc6-4851-4992-b4ab-12c3ad378dfe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"
Sep 07 09:07:08.936568 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.936509    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c18adc6-4851-4992-b4ab-12c3ad378dfe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-8rrhb\" (UID: \"6c18adc6-4851-4992-b4ab-12c3ad378dfe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"
Sep 07 09:07:08.936636 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.936591    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb706726-28a3-4424-84d8-26636b1b5e03-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bb706726-28a3-4424-84d8-26636b1b5e03" (UID: "bb706726-28a3-4424-84d8-26636b1b5e03"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:07:08.938516 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.938485    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb706726-28a3-4424-84d8-26636b1b5e03-kube-api-access-9877p" (OuterVolumeSpecName: "kube-api-access-9877p") pod "bb706726-28a3-4424-84d8-26636b1b5e03" (UID: "bb706726-28a3-4424-84d8-26636b1b5e03"). InnerVolumeSpecName "kube-api-access-9877p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:07:08.940919 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.940899    2574 scope.go:117] "RemoveContainer" containerID="69cd8c89c0dcef9aa6b997b651ae183d1afed407e0879980aeb54fc99bad7146"
Sep 07 09:07:08.941213 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:07:08.941191    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69cd8c89c0dcef9aa6b997b651ae183d1afed407e0879980aeb54fc99bad7146\": container with ID starting with 69cd8c89c0dcef9aa6b997b651ae183d1afed407e0879980aeb54fc99bad7146 not found: ID does not exist" containerID="69cd8c89c0dcef9aa6b997b651ae183d1afed407e0879980aeb54fc99bad7146"
Sep 07 09:07:08.941279 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:08.941225    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69cd8c89c0dcef9aa6b997b651ae183d1afed407e0879980aeb54fc99bad7146"} err="failed to get container status \"69cd8c89c0dcef9aa6b997b651ae183d1afed407e0879980aeb54fc99bad7146\": rpc error: code = NotFound desc = could not find container \"69cd8c89c0dcef9aa6b997b651ae183d1afed407e0879980aeb54fc99bad7146\": container with ID starting with 69cd8c89c0dcef9aa6b997b651ae183d1afed407e0879980aeb54fc99bad7146 not found: ID does not exist"
Sep 07 09:07:09.007214 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.007190    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:09.037854 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.037828    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c18adc6-4851-4992-b4ab-12c3ad378dfe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-8rrhb\" (UID: \"6c18adc6-4851-4992-b4ab-12c3ad378dfe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"
Sep 07 09:07:09.037946 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.037880    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5xrv\" (UniqueName: \"kubernetes.io/projected/6c18adc6-4851-4992-b4ab-12c3ad378dfe-kube-api-access-v5xrv\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-8rrhb\" (UID: \"6c18adc6-4851-4992-b4ab-12c3ad378dfe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"
Sep 07 09:07:09.037946 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.037919    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb706726-28a3-4424-84d8-26636b1b5e03-extensions-socket-volume\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:07:09.037946 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.037929    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9877p\" (UniqueName: \"kubernetes.io/projected/bb706726-28a3-4424-84d8-26636b1b5e03-kube-api-access-9877p\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:07:09.038181 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.038164    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c18adc6-4851-4992-b4ab-12c3ad378dfe-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-8rrhb\" (UID: \"6c18adc6-4851-4992-b4ab-12c3ad378dfe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"
Sep 07 09:07:09.045747 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.045727    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5xrv\" (UniqueName: \"kubernetes.io/projected/6c18adc6-4851-4992-b4ab-12c3ad378dfe-kube-api-access-v5xrv\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-8rrhb\" (UID: \"6c18adc6-4851-4992-b4ab-12c3ad378dfe\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"
Sep 07 09:07:09.110083 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.110058    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:09.144039 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.143991    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"
Sep 07 09:07:09.252911 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.252884    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"]
Sep 07 09:07:09.258951 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.258927    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-db8cx"]
Sep 07 09:07:09.442302 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.442245    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"]
Sep 07 09:07:09.444685 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:07:09.444663    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c18adc6_4851_4992_b4ab_12c3ad378dfe.slice/crio-c6cfcf7543aabbb34f3cfb233472e0638a91cc48da85bbad7ec854b764903717 WatchSource:0}: Error finding container c6cfcf7543aabbb34f3cfb233472e0638a91cc48da85bbad7ec854b764903717: Status 404 returned error can't find the container with id c6cfcf7543aabbb34f3cfb233472e0638a91cc48da85bbad7ec854b764903717
Sep 07 09:07:09.542682 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.542653    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb706726-28a3-4424-84d8-26636b1b5e03" path="/var/lib/kubelet/pods/bb706726-28a3-4424-84d8-26636b1b5e03/volumes"
Sep 07 09:07:09.938525 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.938487    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb" event={"ID":"6c18adc6-4851-4992-b4ab-12c3ad378dfe","Type":"ContainerStarted","Data":"9ceea972a9f8e27b80f024de8ea431b6b22f8c02a71d8f7030c3bdc7d8848640"}
Sep 07 09:07:09.939011 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.938544    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb" event={"ID":"6c18adc6-4851-4992-b4ab-12c3ad378dfe","Type":"ContainerStarted","Data":"c6cfcf7543aabbb34f3cfb233472e0638a91cc48da85bbad7ec854b764903717"}
Sep 07 09:07:09.939011 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.938591    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"
Sep 07 09:07:09.956034 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:09.955980    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb" podStartSLOduration=1.955964966 podStartE2EDuration="1.955964966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:07:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:07:09.954599209 +0000 UTC m=+512.918515430" watchObservedRunningTime="2026-09-07 09:07:09.955964966 +0000 UTC m=+512.919881186"
Sep 07 09:07:10.520617 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:10.520585    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:10.621152 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:10.621117    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:14.790157 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:14.790122    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:14.846678 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:14.846620    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:15.185310 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:15.185278    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:15.286316 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:15.286270    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:15.588243 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:15.588154    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:15.646657 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:15.646617    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:20.946846 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:20.946817    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"
Sep 07 09:07:24.905266 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:24.905231    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:25.339306 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:25.339226    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:40.099250 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.099216    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:40.215554 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.215520    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:40.265357 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.265323    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:40.472221 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.472188    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:07:40.472643 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.472626    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb706726-28a3-4424-84d8-26636b1b5e03" containerName="manager"
Sep 07 09:07:40.472767 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.472648    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb706726-28a3-4424-84d8-26636b1b5e03" containerName="manager"
Sep 07 09:07:40.472767 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.472748    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb706726-28a3-4424-84d8-26636b1b5e03" containerName="manager"
Sep 07 09:07:40.477108 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.477089    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"
Sep 07 09:07:40.479050 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.479026    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 07 09:07:40.483011 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.482988    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:07:40.505947 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.505924    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1d50742b-f01e-4c3e-9743-446f2e216402-config-file\") pod \"limitador-limitador-85d9db4d5d-lwrsj\" (UID: \"1d50742b-f01e-4c3e-9743-446f2e216402\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"
Sep 07 09:07:40.506076 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.505972    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbsp6\" (UniqueName: \"kubernetes.io/projected/1d50742b-f01e-4c3e-9743-446f2e216402-kube-api-access-hbsp6\") pod \"limitador-limitador-85d9db4d5d-lwrsj\" (UID: \"1d50742b-f01e-4c3e-9743-446f2e216402\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"
Sep 07 09:07:40.517571 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.517549    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:07:40.607222 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.607193    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbsp6\" (UniqueName: \"kubernetes.io/projected/1d50742b-f01e-4c3e-9743-446f2e216402-kube-api-access-hbsp6\") pod \"limitador-limitador-85d9db4d5d-lwrsj\" (UID: \"1d50742b-f01e-4c3e-9743-446f2e216402\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"
Sep 07 09:07:40.607364 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.607276    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1d50742b-f01e-4c3e-9743-446f2e216402-config-file\") pod \"limitador-limitador-85d9db4d5d-lwrsj\" (UID: \"1d50742b-f01e-4c3e-9743-446f2e216402\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"
Sep 07 09:07:40.607851 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.607834    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1d50742b-f01e-4c3e-9743-446f2e216402-config-file\") pod \"limitador-limitador-85d9db4d5d-lwrsj\" (UID: \"1d50742b-f01e-4c3e-9743-446f2e216402\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"
Sep 07 09:07:40.614896 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.614871    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbsp6\" (UniqueName: \"kubernetes.io/projected/1d50742b-f01e-4c3e-9743-446f2e216402-kube-api-access-hbsp6\") pod \"limitador-limitador-85d9db4d5d-lwrsj\" (UID: \"1d50742b-f01e-4c3e-9743-446f2e216402\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"
Sep 07 09:07:40.789215 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.789137    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"
Sep 07 09:07:40.925353 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:40.925323    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:41.021612 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:41.021577    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:41.061943 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:41.061912    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:07:41.064545 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:07:41.064509    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d50742b_f01e_4c3e_9743_446f2e216402.slice/crio-19e9cec85abf8caebc4ec64ff5dea283b45dc62002816689b4a347a3b07b76a9 WatchSource:0}: Error finding container 19e9cec85abf8caebc4ec64ff5dea283b45dc62002816689b4a347a3b07b76a9: Status 404 returned error can't find the container with id 19e9cec85abf8caebc4ec64ff5dea283b45dc62002816689b4a347a3b07b76a9
Sep 07 09:07:41.097336 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:41.097306    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj" event={"ID":"1d50742b-f01e-4c3e-9743-446f2e216402","Type":"ContainerStarted","Data":"19e9cec85abf8caebc4ec64ff5dea283b45dc62002816689b4a347a3b07b76a9"}
Sep 07 09:07:42.103854 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:42.103775    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj" event={"ID":"1d50742b-f01e-4c3e-9743-446f2e216402","Type":"ContainerStarted","Data":"c1d54541bf4ad6ef62f6ef1cbe3e31b96cdffae41b471d98d810fdc261996d94"}
Sep 07 09:07:42.104184 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:42.103890    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"
Sep 07 09:07:42.120562 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:42.120515    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj" podStartSLOduration=1.349537977 podStartE2EDuration="2.120501905s" totalImagesPullingTime="770.963928ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:07:42.118258479 +0000 UTC m=+545.082174700" watchObservedRunningTime="2026-09-07 09:07:42.120501905 +0000 UTC m=+545.084418124"
Sep 07 09:07:53.109920 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:53.109891    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"
Sep 07 09:07:58.750093 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:58.750058    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:58.849112 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:58.849078    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:07:59.558819 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:07:59.558790    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:12.231497 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:12.231464    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:12.477421 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:12.477383    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:12.558630 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:12.556076    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-687bd45585-g5mgg"]
Sep 07 09:08:12.560896 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:12.560862    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-g5mgg"
Sep 07 09:08:12.563737 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:12.563440    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hp9wv\""
Sep 07 09:08:12.563737 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:12.563444    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 07 09:08:12.565953 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:12.565931    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-687bd45585-g5mgg"]
Sep 07 09:08:12.666615 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:12.666583    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7a7cf8f1-cbb9-476a-9124-5ddc6e07853a-tls-cert\") pod \"authorino-687bd45585-g5mgg\" (UID: \"7a7cf8f1-cbb9-476a-9124-5ddc6e07853a\") " pod="kuadrant-system/authorino-687bd45585-g5mgg"
Sep 07 09:08:12.666615 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:12.666618    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wfjz\" (UniqueName: \"kubernetes.io/projected/7a7cf8f1-cbb9-476a-9124-5ddc6e07853a-kube-api-access-4wfjz\") pod \"authorino-687bd45585-g5mgg\" (UID: \"7a7cf8f1-cbb9-476a-9124-5ddc6e07853a\") " pod="kuadrant-system/authorino-687bd45585-g5mgg"
Sep 07 09:08:12.680504 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:12.680476    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-g5mgg"]
Sep 07 09:08:12.680737 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:08:12.680704    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-4wfjz tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-687bd45585-g5mgg" podUID="7a7cf8f1-cbb9-476a-9124-5ddc6e07853a"
Sep 07 09:08:12.767052 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:12.767020    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7a7cf8f1-cbb9-476a-9124-5ddc6e07853a-tls-cert\") pod \"authorino-687bd45585-g5mgg\" (UID: \"7a7cf8f1-cbb9-476a-9124-5ddc6e07853a\") " pod="kuadrant-system/authorino-687bd45585-g5mgg"
Sep 07 09:08:12.767052 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:12.767053    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wfjz\" (UniqueName: \"kubernetes.io/projected/7a7cf8f1-cbb9-476a-9124-5ddc6e07853a-kube-api-access-4wfjz\") pod \"authorino-687bd45585-g5mgg\" (UID: \"7a7cf8f1-cbb9-476a-9124-5ddc6e07853a\") " pod="kuadrant-system/authorino-687bd45585-g5mgg"
Sep 07 09:08:12.769737 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:12.769690    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7a7cf8f1-cbb9-476a-9124-5ddc6e07853a-tls-cert\") pod \"authorino-687bd45585-g5mgg\" (UID: \"7a7cf8f1-cbb9-476a-9124-5ddc6e07853a\") " pod="kuadrant-system/authorino-687bd45585-g5mgg"
Sep 07 09:08:12.774509 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:12.774484    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wfjz\" (UniqueName: \"kubernetes.io/projected/7a7cf8f1-cbb9-476a-9124-5ddc6e07853a-kube-api-access-4wfjz\") pod \"authorino-687bd45585-g5mgg\" (UID: \"7a7cf8f1-cbb9-476a-9124-5ddc6e07853a\") " pod="kuadrant-system/authorino-687bd45585-g5mgg"
Sep 07 09:08:13.259480 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:13.259451    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-g5mgg"
Sep 07 09:08:13.268129 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:13.268109    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-g5mgg"
Sep 07 09:08:13.371455 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:13.371423    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wfjz\" (UniqueName: \"kubernetes.io/projected/7a7cf8f1-cbb9-476a-9124-5ddc6e07853a-kube-api-access-4wfjz\") pod \"7a7cf8f1-cbb9-476a-9124-5ddc6e07853a\" (UID: \"7a7cf8f1-cbb9-476a-9124-5ddc6e07853a\") "
Sep 07 09:08:13.371613 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:13.371514    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7a7cf8f1-cbb9-476a-9124-5ddc6e07853a-tls-cert\") pod \"7a7cf8f1-cbb9-476a-9124-5ddc6e07853a\" (UID: \"7a7cf8f1-cbb9-476a-9124-5ddc6e07853a\") "
Sep 07 09:08:13.373698 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:13.373663    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a7cf8f1-cbb9-476a-9124-5ddc6e07853a-kube-api-access-4wfjz" (OuterVolumeSpecName: "kube-api-access-4wfjz") pod "7a7cf8f1-cbb9-476a-9124-5ddc6e07853a" (UID: "7a7cf8f1-cbb9-476a-9124-5ddc6e07853a"). InnerVolumeSpecName "kube-api-access-4wfjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:08:13.373698 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:13.373674    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a7cf8f1-cbb9-476a-9124-5ddc6e07853a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "7a7cf8f1-cbb9-476a-9124-5ddc6e07853a" (UID: "7a7cf8f1-cbb9-476a-9124-5ddc6e07853a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:08:13.472212 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:13.472187    2574 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7a7cf8f1-cbb9-476a-9124-5ddc6e07853a-tls-cert\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:08:13.472212 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:13.472209    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wfjz\" (UniqueName: \"kubernetes.io/projected/7a7cf8f1-cbb9-476a-9124-5ddc6e07853a-kube-api-access-4wfjz\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:08:14.264431 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:14.264394    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-g5mgg"
Sep 07 09:08:14.287127 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:14.287100    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-g5mgg"]
Sep 07 09:08:14.290849 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:14.290826    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-687bd45585-g5mgg"]
Sep 07 09:08:15.541894 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:15.541851    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a7cf8f1-cbb9-476a-9124-5ddc6e07853a" path="/var/lib/kubelet/pods/7a7cf8f1-cbb9-476a-9124-5ddc6e07853a/volumes"
Sep 07 09:08:21.641287 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:21.641251    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:21.715076 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:21.715034    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:21.776428 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:21.776371    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:21.906075 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:21.906041    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:21.960382 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:21.960345    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:25.116527 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:25.116488    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:25.216803 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:25.216772    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:35.776125 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:35.776089    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:37.475519 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:37.475490    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:08:37.476409 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:37.476391    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:08:37.483018 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:37.482995    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:08:37.483937 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:37.483920    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:08:57.715141 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:57.715074    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:58.890800 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:58.890693    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:59.049068 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:59.049036    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:59.149287 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:59.149176    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:59.316799 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:59.316762    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:59.389407 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:59.389363    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:59.587694 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:59.587655    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:59.681153 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:59.681110    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:08:59.723790 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:08:59.723753    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:09:11.179487 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:11.179446    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:09:11.286948 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:11.286919    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:09:27.000346 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.000312    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"]
Sep 07 09:09:27.003015 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.002999    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.005418 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.005389    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 07 09:09:27.005418 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.005409    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5tt6q\""
Sep 07 09:09:27.005418 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.005413    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 07 09:09:27.005623 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.005465    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 07 09:09:27.011461 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.011443    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"]
Sep 07 09:09:27.168812 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.168781    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6c6b323-2416-44f8-bd0d-c31db8070ef5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.168961 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.168835    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6c6b323-2416-44f8-bd0d-c31db8070ef5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.168961 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.168928    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6c6b323-2416-44f8-bd0d-c31db8070ef5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.169049 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.168999    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6c6b323-2416-44f8-bd0d-c31db8070ef5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.169049 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.169018    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6c6b323-2416-44f8-bd0d-c31db8070ef5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.169049 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.169041    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdmvz\" (UniqueName: \"kubernetes.io/projected/f6c6b323-2416-44f8-bd0d-c31db8070ef5-kube-api-access-pdmvz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.192628 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.192584    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:09:27.269947 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.269860    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6c6b323-2416-44f8-bd0d-c31db8070ef5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.269947 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.269916    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6c6b323-2416-44f8-bd0d-c31db8070ef5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.270154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.269961    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6c6b323-2416-44f8-bd0d-c31db8070ef5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.270154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.269978    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6c6b323-2416-44f8-bd0d-c31db8070ef5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.270154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.269998    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdmvz\" (UniqueName: \"kubernetes.io/projected/f6c6b323-2416-44f8-bd0d-c31db8070ef5-kube-api-access-pdmvz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.270154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.270026    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6c6b323-2416-44f8-bd0d-c31db8070ef5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.270358 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.270289    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6c6b323-2416-44f8-bd0d-c31db8070ef5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.270358 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.270312    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6c6b323-2416-44f8-bd0d-c31db8070ef5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.270463 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.270421    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6c6b323-2416-44f8-bd0d-c31db8070ef5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.272460 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.272438    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6c6b323-2416-44f8-bd0d-c31db8070ef5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.272580 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.272566    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6c6b323-2416-44f8-bd0d-c31db8070ef5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.278102 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.278075    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdmvz\" (UniqueName: \"kubernetes.io/projected/f6c6b323-2416-44f8-bd0d-c31db8070ef5-kube-api-access-pdmvz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44\" (UID: \"f6c6b323-2416-44f8-bd0d-c31db8070ef5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.313648 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.313626    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:27.588536 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.584424    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"]
Sep 07 09:09:27.590206 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:09:27.590175    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6c6b323_2416_44f8_bd0d_c31db8070ef5.slice/crio-9a70771f9620d32f9c06773ae9dca10d85a04a69a9c8bab115e506640ebfbd97 WatchSource:0}: Error finding container 9a70771f9620d32f9c06773ae9dca10d85a04a69a9c8bab115e506640ebfbd97: Status 404 returned error can't find the container with id 9a70771f9620d32f9c06773ae9dca10d85a04a69a9c8bab115e506640ebfbd97
Sep 07 09:09:27.613770 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.613740    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44" event={"ID":"f6c6b323-2416-44f8-bd0d-c31db8070ef5","Type":"ContainerStarted","Data":"9a70771f9620d32f9c06773ae9dca10d85a04a69a9c8bab115e506640ebfbd97"}
Sep 07 09:09:27.675164 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:27.675130    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:09:29.625097 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:29.625060    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44" event={"ID":"f6c6b323-2416-44f8-bd0d-c31db8070ef5","Type":"ContainerStarted","Data":"5218ad0d70b3514eee8a0c1f1e86619f6809826c4f4fc654e380b109d5029d49"}
Sep 07 09:09:29.641733 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:29.641669    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44" podStartSLOduration=2.5443070519999997 podStartE2EDuration="3.641651982s" totalImagesPullingTime="1.09734493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:09:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:09:29.641121509 +0000 UTC m=+652.605037746" watchObservedRunningTime="2026-09-07 09:09:29.641651982 +0000 UTC m=+652.605568197"
Sep 07 09:09:37.314179 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:37.314145    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:37.314574 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:37.314199    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:37.326537 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:37.326512    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:37.676847 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:37.676822    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44"
Sep 07 09:09:44.331128 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:44.331085    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:09:45.000255 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:45.000214    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:09:56.695549 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.695516    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"]
Sep 07 09:09:56.699078 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.699060    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.701074 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.701049    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 07 09:09:56.707841 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.707816    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"]
Sep 07 09:09:56.725458 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.725429    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:09:56.807191 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.807158    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c88817c0-4c0e-4acd-96e4-fa3247282e41-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.807361 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.807203    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c88817c0-4c0e-4acd-96e4-fa3247282e41-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.807361 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.807224    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c88817c0-4c0e-4acd-96e4-fa3247282e41-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.807361 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.807243    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c88817c0-4c0e-4acd-96e4-fa3247282e41-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.807361 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.807277    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c88817c0-4c0e-4acd-96e4-fa3247282e41-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.807361 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.807294    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5sg2\" (UniqueName: \"kubernetes.io/projected/c88817c0-4c0e-4acd-96e4-fa3247282e41-kube-api-access-f5sg2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.908623 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.908587    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c88817c0-4c0e-4acd-96e4-fa3247282e41-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.908840 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.908639    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c88817c0-4c0e-4acd-96e4-fa3247282e41-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.908840 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.908656    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c88817c0-4c0e-4acd-96e4-fa3247282e41-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.908840 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.908674    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c88817c0-4c0e-4acd-96e4-fa3247282e41-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.908840 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.908727    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c88817c0-4c0e-4acd-96e4-fa3247282e41-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.908840 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.908747    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5sg2\" (UniqueName: \"kubernetes.io/projected/c88817c0-4c0e-4acd-96e4-fa3247282e41-kube-api-access-f5sg2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.909100 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.909074    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c88817c0-4c0e-4acd-96e4-fa3247282e41-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.909193 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.909172    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c88817c0-4c0e-4acd-96e4-fa3247282e41-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.909250 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.909180    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c88817c0-4c0e-4acd-96e4-fa3247282e41-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.911189 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.911169    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c88817c0-4c0e-4acd-96e4-fa3247282e41-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.911342 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.911323    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c88817c0-4c0e-4acd-96e4-fa3247282e41-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:56.916122 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:56.916097    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5sg2\" (UniqueName: \"kubernetes.io/projected/c88817c0-4c0e-4acd-96e4-fa3247282e41-kube-api-access-f5sg2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4\" (UID: \"c88817c0-4c0e-4acd-96e4-fa3247282e41\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:57.010207 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:57.010121    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:09:57.279995 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:57.279967    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"]
Sep 07 09:09:57.282175 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:09:57.282149    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc88817c0_4c0e_4acd_96e4_fa3247282e41.slice/crio-6331e56bd8f7e1d007457192004eb6f290e3b66f4926ac41ac3a07da65e1e239 WatchSource:0}: Error finding container 6331e56bd8f7e1d007457192004eb6f290e3b66f4926ac41ac3a07da65e1e239: Status 404 returned error can't find the container with id 6331e56bd8f7e1d007457192004eb6f290e3b66f4926ac41ac3a07da65e1e239
Sep 07 09:09:57.283887 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:57.283870    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 09:09:57.604502 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:57.604466    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:09:57.762461 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:57.762423    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4" event={"ID":"c88817c0-4c0e-4acd-96e4-fa3247282e41","Type":"ContainerStarted","Data":"9c159b52a74ea6f4004d5d6f47d979132fc694e80a57f77ed9fe5f61257eca3f"}
Sep 07 09:09:57.762923 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:57.762478    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4" event={"ID":"c88817c0-4c0e-4acd-96e4-fa3247282e41","Type":"ContainerStarted","Data":"6331e56bd8f7e1d007457192004eb6f290e3b66f4926ac41ac3a07da65e1e239"}
Sep 07 09:09:57.777669 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:09:57.777625    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4" podStartSLOduration=1.566534594 podStartE2EDuration="1.777608975s" totalImagesPullingTime="211.074381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:09:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:09:57.77632542 +0000 UTC m=+680.740241661" watchObservedRunningTime="2026-09-07 09:09:57.777608975 +0000 UTC m=+680.741525195"
Sep 07 09:10:07.010983 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:07.010933    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:10:07.011418 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:07.011001    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:10:07.023505 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:07.023474    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:10:07.819869 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:07.819839    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4"
Sep 07 09:10:17.128664 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:17.128628    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:10:18.197340 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:18.197306    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:10:26.707210 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.707181    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"]
Sep 07 09:10:26.709584 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.709569    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.711566 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.711536    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 07 09:10:26.718666 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.718646    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"]
Sep 07 09:10:26.728740 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.728695    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:10:26.866607 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.866572    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89aeaf5e-978b-4fea-b38c-d75709fdce2a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.866763 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.866615    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89aeaf5e-978b-4fea-b38c-d75709fdce2a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.866763 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.866641    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89aeaf5e-978b-4fea-b38c-d75709fdce2a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.866763 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.866701    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlbcp\" (UniqueName: \"kubernetes.io/projected/89aeaf5e-978b-4fea-b38c-d75709fdce2a-kube-api-access-jlbcp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.866763 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.866748    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89aeaf5e-978b-4fea-b38c-d75709fdce2a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.866957 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.866834    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89aeaf5e-978b-4fea-b38c-d75709fdce2a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.968126 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.968048    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89aeaf5e-978b-4fea-b38c-d75709fdce2a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.968126 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.968105    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89aeaf5e-978b-4fea-b38c-d75709fdce2a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.968301 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.968136    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89aeaf5e-978b-4fea-b38c-d75709fdce2a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.968301 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.968162    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89aeaf5e-978b-4fea-b38c-d75709fdce2a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.968301 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.968207    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlbcp\" (UniqueName: \"kubernetes.io/projected/89aeaf5e-978b-4fea-b38c-d75709fdce2a-kube-api-access-jlbcp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.968301 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.968234    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89aeaf5e-978b-4fea-b38c-d75709fdce2a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.968551 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.968532    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89aeaf5e-978b-4fea-b38c-d75709fdce2a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.968620 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.968562    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89aeaf5e-978b-4fea-b38c-d75709fdce2a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.968620 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.968608    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89aeaf5e-978b-4fea-b38c-d75709fdce2a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.970571 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.970551    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89aeaf5e-978b-4fea-b38c-d75709fdce2a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.970848 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.970829    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89aeaf5e-978b-4fea-b38c-d75709fdce2a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:26.974912 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:26.974893    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlbcp\" (UniqueName: \"kubernetes.io/projected/89aeaf5e-978b-4fea-b38c-d75709fdce2a-kube-api-access-jlbcp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb\" (UID: \"89aeaf5e-978b-4fea-b38c-d75709fdce2a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:27.021623 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:27.021602    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:27.290321 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:27.290298    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"]
Sep 07 09:10:27.292675 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:10:27.292646    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89aeaf5e_978b_4fea_b38c_d75709fdce2a.slice/crio-45319695f151dd66bbc211014dd6b474f9a9a7faf8bbffae19a94349ff93c24a WatchSource:0}: Error finding container 45319695f151dd66bbc211014dd6b474f9a9a7faf8bbffae19a94349ff93c24a: Status 404 returned error can't find the container with id 45319695f151dd66bbc211014dd6b474f9a9a7faf8bbffae19a94349ff93c24a
Sep 07 09:10:27.907500 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:27.907463    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb" event={"ID":"89aeaf5e-978b-4fea-b38c-d75709fdce2a","Type":"ContainerStarted","Data":"a186c55afb33abb8a956e38f2679338a3b61ca9a506c48003728ddb5243dffa6"}
Sep 07 09:10:27.907932 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:27.907521    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb" event={"ID":"89aeaf5e-978b-4fea-b38c-d75709fdce2a","Type":"ContainerStarted","Data":"45319695f151dd66bbc211014dd6b474f9a9a7faf8bbffae19a94349ff93c24a"}
Sep 07 09:10:27.923274 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:27.923226    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb" podStartSLOduration=1.758742681 podStartE2EDuration="1.923209075s" totalImagesPullingTime="164.466394ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:10:27.922335585 +0000 UTC m=+710.886251835" watchObservedRunningTime="2026-09-07 09:10:27.923209075 +0000 UTC m=+710.887125297"
Sep 07 09:10:28.405487 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:28.405458    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:10:37.022654 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:37.022620    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:37.023299 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:37.022678    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:37.035084 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:37.035057    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:37.967794 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:37.967765    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb"
Sep 07 09:10:42.839062 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:42.839016    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:10:44.399592 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:10:44.399565    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:11:02.703932 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.703901    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"]
Sep 07 09:11:02.706846 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.706819    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.708923 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.708895    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 07 09:11:02.718466 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.718438    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"]
Sep 07 09:11:02.738924 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.738899    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:11:02.763888 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.763867    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfb234f0-e006-4d23-bda0-5cbed8f93617-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.764010 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.763897    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfb234f0-e006-4d23-bda0-5cbed8f93617-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.764010 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.763924    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfb234f0-e006-4d23-bda0-5cbed8f93617-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.764010 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.763948    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfb234f0-e006-4d23-bda0-5cbed8f93617-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.764122 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.764017    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hxpf\" (UniqueName: \"kubernetes.io/projected/dfb234f0-e006-4d23-bda0-5cbed8f93617-kube-api-access-5hxpf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.764122 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.764054    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfb234f0-e006-4d23-bda0-5cbed8f93617-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.865051 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.865019    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfb234f0-e006-4d23-bda0-5cbed8f93617-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.865213 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.865070    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfb234f0-e006-4d23-bda0-5cbed8f93617-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.865213 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.865187    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfb234f0-e006-4d23-bda0-5cbed8f93617-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.865302 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.865241    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfb234f0-e006-4d23-bda0-5cbed8f93617-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.865302 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.865270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfb234f0-e006-4d23-bda0-5cbed8f93617-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.865390 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.865343    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hxpf\" (UniqueName: \"kubernetes.io/projected/dfb234f0-e006-4d23-bda0-5cbed8f93617-kube-api-access-5hxpf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.865487 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.865468    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfb234f0-e006-4d23-bda0-5cbed8f93617-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.865549 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.865499    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfb234f0-e006-4d23-bda0-5cbed8f93617-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.865743 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.865699    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfb234f0-e006-4d23-bda0-5cbed8f93617-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.867485 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.867470    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfb234f0-e006-4d23-bda0-5cbed8f93617-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.867601 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.867585    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfb234f0-e006-4d23-bda0-5cbed8f93617-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:02.872164 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:02.872144    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hxpf\" (UniqueName: \"kubernetes.io/projected/dfb234f0-e006-4d23-bda0-5cbed8f93617-kube-api-access-5hxpf\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5\" (UID: \"dfb234f0-e006-4d23-bda0-5cbed8f93617\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:03.016878 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:03.016792    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:03.288103 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:03.288027    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"]
Sep 07 09:11:03.291436 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:11:03.291397    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfb234f0_e006_4d23_bda0_5cbed8f93617.slice/crio-35b3034e8b24d11b5edb79aa7d1180aa063a761b84356accbf88a5b02c9f977f WatchSource:0}: Error finding container 35b3034e8b24d11b5edb79aa7d1180aa063a761b84356accbf88a5b02c9f977f: Status 404 returned error can't find the container with id 35b3034e8b24d11b5edb79aa7d1180aa063a761b84356accbf88a5b02c9f977f
Sep 07 09:11:04.085396 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:04.085366    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5" event={"ID":"dfb234f0-e006-4d23-bda0-5cbed8f93617","Type":"ContainerStarted","Data":"e84e16580ad512a4131cf0ee5f3e745c979ba2fb32fc063d28dd3e834599bd87"}
Sep 07 09:11:04.085787 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:04.085410    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5" event={"ID":"dfb234f0-e006-4d23-bda0-5cbed8f93617","Type":"ContainerStarted","Data":"35b3034e8b24d11b5edb79aa7d1180aa063a761b84356accbf88a5b02c9f977f"}
Sep 07 09:11:04.100042 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:04.099995    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5" podStartSLOduration=1.923893149 podStartE2EDuration="2.099979289s" totalImagesPullingTime="176.08614ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:11:04.099178147 +0000 UTC m=+747.063094391" watchObservedRunningTime="2026-09-07 09:11:04.099979289 +0000 UTC m=+747.063895559"
Sep 07 09:11:04.700299 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:04.700268    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:11:13.017632 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:13.017600    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:13.017632 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:13.017639    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:13.029772 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:13.029754    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:11:13.138298 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:11:13.138266    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5"
Sep 07 09:12:30.674809 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:30.674758    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:30.773056 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:30.773027    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:31.717423 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:31.717226    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:31.815144 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:31.815113    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:47.364596 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:47.364555    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:47.465534 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:47.465499    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:47.643921 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:47.643839    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:47.933758 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:47.933730    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:48.696285 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:48.696249    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:48.792104 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:48.792072    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:49.135231 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:49.135147    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:50.757980 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:50.757948    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:12:53.340360 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:53.340326    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:53.423017 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:53.422991    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:54.489345 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:54.489315    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:54.583036 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:54.583000    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:54.652114 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:54.652083    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:54.955526 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:54.955497    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:55.094194 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:55.094163    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:55.858412 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:55.858378    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:55.958380 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:55.958342    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:56.072896 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:56.072858    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:56.358878 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:56.358843    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:56.614418 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:56.614326    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:56.968776 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:56.968745    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:57.236800 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:57.236723    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:57.426166 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:57.426137    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:58.524982 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:58.524948    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:58.618825 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:58.618799    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:59.660802 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:59.660769    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:59.764345 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:59.764304    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:59.860779 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:59.860742    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:12:59.985828 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:12:59.985700    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:00.084571 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:00.084539    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:00.186818 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:00.186779    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:00.284371 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:00.284283    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:01.039380 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.039350    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:13:01.202100 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.202071    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"]
Sep 07 09:13:01.205390 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.205369    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:01.207612 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.207580    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 09:13:01.207801 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.207586    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-da4d4c-serving-cert\""
Sep 07 09:13:01.207801 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.207596    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 07 09:13:01.207981 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.207659    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 07 09:13:01.208248 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.208225    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rckvc\""
Sep 07 09:13:01.221323 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.221301    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"]
Sep 07 09:13:01.247305 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.247279    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:01.339490 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.339373    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:01.372507 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.372481    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-openshift-service-ca\") pod \"maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:01.372622 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.372511    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-maas-api-tls\") pod \"maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:01.372622 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.372562    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4ds8\" (UniqueName: \"kubernetes.io/projected/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-kube-api-access-d4ds8\") pod \"maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:01.372622 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.372609    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-metrics-certs\") pod \"maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:01.404286 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.404263    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:01.473278 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.473241    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-openshift-service-ca\") pod \"maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:01.473278 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.473279    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-maas-api-tls\") pod \"maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:01.473513 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.473335    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4ds8\" (UniqueName: \"kubernetes.io/projected/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-kube-api-access-d4ds8\") pod \"maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:01.473513 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.473386    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-metrics-certs\") pod \"maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:01.473978 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.473956    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-openshift-service-ca\") pod \"maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:01.476018 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.476000    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-maas-api-tls\") pod \"maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:01.476103 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.476041    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-metrics-certs\") pod \"maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:01.480945 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.480927    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4ds8\" (UniqueName: \"kubernetes.io/projected/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-kube-api-access-d4ds8\") pod \"maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:01.516986 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.516953    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:01.682124 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.682095    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:01.781048 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.781013    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:01.788303 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:01.788279    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"]
Sep 07 09:13:01.790888 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:13:01.790860    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c4bfcc_a77c_4bd5_ac3b_ff00a39c5403.slice/crio-c73e44765f16a8c35bc415efcdb52399cac585ec0dba47343febc521d6051eab WatchSource:0}: Error finding container c73e44765f16a8c35bc415efcdb52399cac585ec0dba47343febc521d6051eab: Status 404 returned error can't find the container with id c73e44765f16a8c35bc415efcdb52399cac585ec0dba47343febc521d6051eab
Sep 07 09:13:02.146638 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:02.146549    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:02.659335 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:02.659300    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4" event={"ID":"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403","Type":"ContainerStarted","Data":"c73e44765f16a8c35bc415efcdb52399cac585ec0dba47343febc521d6051eab"}
Sep 07 09:13:02.971388 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:02.971304    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:04.671886 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:04.671850    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4" event={"ID":"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403","Type":"ContainerStarted","Data":"b8da24f234335b8abc6c59d25456c3e2b9d5fe20fc89144710146c4b0b3d28ad"}
Sep 07 09:13:04.672290 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:04.671945    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:04.689114 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:04.689069    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4" podStartSLOduration=1.332496404 podStartE2EDuration="3.689053863s" totalImagesPullingTime="2.356557459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:13:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:04.687868344 +0000 UTC m=+867.651784583" watchObservedRunningTime="2026-09-07 09:13:04.689053863 +0000 UTC m=+867.652970129"
Sep 07 09:13:05.788048 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:05.788006    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:05.947910 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:05.947867    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:06.043797 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:06.043668    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:06.147477 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:06.147445    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:06.243377 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:06.243338    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:08.789445 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:08.789410    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:10.380231 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:10.380195    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:10.683513 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:10.683487    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:13:11.991455 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:11.991425    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:12.092629 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:12.092590    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:12.191520 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:12.191489    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:12.393010 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:12.392928    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:13.613228 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.613203    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"]
Sep 07 09:13:13.615909 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.615891    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.617872 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.617855    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-9d926599-kserve-self-signed-certs\""
Sep 07 09:13:13.627439 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.627418    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"]
Sep 07 09:13:13.681008 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.680985    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25292215-7d1f-4930-adcf-85cffc684ef7-tls-certs\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.681142 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.681031    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-dshm\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.681142 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.681095    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-model-cache\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.681142 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.681130    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-tmp-dir\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.681296 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.681152    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-home\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.681296 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.681165    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lffmd\" (UniqueName: \"kubernetes.io/projected/25292215-7d1f-4930-adcf-85cffc684ef7-kube-api-access-lffmd\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.698840 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.698821    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:13:13.782627 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.782586    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-dshm\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.782815 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.782642    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-model-cache\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.782815 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.782681    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-tmp-dir\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.782815 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.782702    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-home\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.782815 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.782743    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lffmd\" (UniqueName: \"kubernetes.io/projected/25292215-7d1f-4930-adcf-85cffc684ef7-kube-api-access-lffmd\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.783028 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.782860    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25292215-7d1f-4930-adcf-85cffc684ef7-tls-certs\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.783672 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.783623    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-tmp-dir\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.785284 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.783864    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-model-cache\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.785544 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.785516    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-dshm\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.787654 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.787634    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-home\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.791130 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.790872    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lffmd\" (UniqueName: \"kubernetes.io/projected/25292215-7d1f-4930-adcf-85cffc684ef7-kube-api-access-lffmd\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.795062 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.795038    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25292215-7d1f-4930-adcf-85cffc684ef7-tls-certs\") pod \"e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") " pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:13.801488 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.801466    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:13.925059 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:13.925034    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:14.195421 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:14.195374    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"]
Sep 07 09:13:14.198197 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:13:14.198159    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25292215_7d1f_4930_adcf_85cffc684ef7.slice/crio-ff858de9db678682c928680a72426c6da7498b2e313965c4c912773fdd0855cd WatchSource:0}: Error finding container ff858de9db678682c928680a72426c6da7498b2e313965c4c912773fdd0855cd: Status 404 returned error can't find the container with id ff858de9db678682c928680a72426c6da7498b2e313965c4c912773fdd0855cd
Sep 07 09:13:14.720498 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:14.720467    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q" event={"ID":"25292215-7d1f-4930-adcf-85cffc684ef7","Type":"ContainerStarted","Data":"1e75f0d65109b0753c0a9a1cdd247f71254089099b710cbf0b6914f483a58e67"}
Sep 07 09:13:14.720911 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:14.720505    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q" event={"ID":"25292215-7d1f-4930-adcf-85cffc684ef7","Type":"ContainerStarted","Data":"ff858de9db678682c928680a72426c6da7498b2e313965c4c912773fdd0855cd"}
Sep 07 09:13:15.424993 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:15.424954    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:18.158965 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:18.158928    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:18.253502 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:18.253452    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:18.320347 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:18.320314    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:18.491813 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:18.491736    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:18.798899 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:18.798816    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:19.409486 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:19.409451    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:20.614190 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:20.614160    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:22.064220 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:22.064182    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:22.222665 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:22.222583    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:22.322734 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:22.322669    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:23.651226 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.651169    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q" podStartSLOduration=10.651148057 podStartE2EDuration="10.651148057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:13:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:14.73447851 +0000 UTC m=+877.698394757" watchObservedRunningTime="2026-09-07 09:13:23.651148057 +0000 UTC m=+886.615064280"
Sep 07 09:13:23.652104 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.652087    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"]
Sep 07 09:13:23.654592 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.654573    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:23.656470 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.656455    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-3bcf58-serving-cert\""
Sep 07 09:13:23.664800 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.664774    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"]
Sep 07 09:13:23.699506 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.699475    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:23.768932 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.768905    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6qj7\" (UniqueName: \"kubernetes.io/projected/2d8be2de-450b-46df-a2cf-37cdf62c02db-kube-api-access-m6qj7\") pod \"maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:23.769058 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.768951    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-maas-api-tls\") pod \"maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:23.769058 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.768972    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-metrics-certs\") pod \"maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:23.769058 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.769013    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d8be2de-450b-46df-a2cf-37cdf62c02db-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:23.799969 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.799941    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:23.870212 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.870188    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6qj7\" (UniqueName: \"kubernetes.io/projected/2d8be2de-450b-46df-a2cf-37cdf62c02db-kube-api-access-m6qj7\") pod \"maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:23.870316 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.870237    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-maas-api-tls\") pod \"maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:23.870366 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:13:23.870331    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-serving-cert: secret "maas-api-e2e-inf-b-3bcf58-serving-cert" not found
Sep 07 09:13:23.870409 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.870364    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-metrics-certs\") pod \"maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:23.870409 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:13:23.870388    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-maas-api-tls podName:2d8be2de-450b-46df-a2cf-37cdf62c02db nodeName:}" failed. No retries permitted until 2026-09-07 09:13:24.370368745 +0000 UTC m=+887.334284961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-maas-api-tls") pod "maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5" (UID: "2d8be2de-450b-46df-a2cf-37cdf62c02db") : secret "maas-api-e2e-inf-b-3bcf58-serving-cert" not found
Sep 07 09:13:23.870521 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.870419    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d8be2de-450b-46df-a2cf-37cdf62c02db-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:23.871102 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.871080    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d8be2de-450b-46df-a2cf-37cdf62c02db-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:23.872889 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.872865    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-metrics-certs\") pod \"maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:23.878303 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.878282    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6qj7\" (UniqueName: \"kubernetes.io/projected/2d8be2de-450b-46df-a2cf-37cdf62c02db-kube-api-access-m6qj7\") pod \"maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:23.925241 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.925219    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:23.925356 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.925284    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:23.940856 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.940831    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:23.989950 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:23.989924    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:24.115644 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:24.115613    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:24.218446 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:24.218366    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:24.375459 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:24.375424    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-maas-api-tls\") pod \"maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:24.378190 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:24.378168    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-maas-api-tls\") pod \"maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:24.568619 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:24.568526    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:24.633812 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:24.633776    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:24.781215 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:24.781188    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:13:24.840379 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:24.840348    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"]
Sep 07 09:13:24.841987 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:13:24.841961    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d8be2de_450b_46df_a2cf_37cdf62c02db.slice/crio-35dc873b3ccec67182b4d2097be9a1ff399eacb243e10e153794e58432f029b1 WatchSource:0}: Error finding container 35dc873b3ccec67182b4d2097be9a1ff399eacb243e10e153794e58432f029b1: Status 404 returned error can't find the container with id 35dc873b3ccec67182b4d2097be9a1ff399eacb243e10e153794e58432f029b1
Sep 07 09:13:24.886832 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:24.886807    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:25.337420 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:25.337379    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:25.437696 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:25.437611    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:25.776754 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:25.776652    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5" event={"ID":"2d8be2de-450b-46df-a2cf-37cdf62c02db","Type":"ContainerStarted","Data":"39519caa8e526d9a04f5672791e1f321cced92ae6599eb0e4b1446cc016db0e7"}
Sep 07 09:13:25.776754 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:25.776732    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5" event={"ID":"2d8be2de-450b-46df-a2cf-37cdf62c02db","Type":"ContainerStarted","Data":"35dc873b3ccec67182b4d2097be9a1ff399eacb243e10e153794e58432f029b1"}
Sep 07 09:13:26.504831 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:26.504768    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5" podStartSLOduration=3.162099176 podStartE2EDuration="3.504750538s" totalImagesPullingTime="342.651362ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:13:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:13:25.792874768 +0000 UTC m=+888.756791017" watchObservedRunningTime="2026-09-07 09:13:26.504750538 +0000 UTC m=+889.468666802"
Sep 07 09:13:26.506082 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:26.506059    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:13:26.781912 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:26.781829    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:32.791559 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:32.791484    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:13:37.506555 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:37.506526    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:13:37.507726 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:37.507693    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:13:37.514253 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:37.514232    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:13:37.515476 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:37.515458    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:13:40.024449 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:40.024409    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:40.553946 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:40.553913    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:40.653774 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:40.653747    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:40.760459 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:40.760427    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:40.862172 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:40.862091    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:40.928368 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:40.928322    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:41.033138 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:41.033102    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:41.225813 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:41.225783    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:41.326777 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:41.326735    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:41.432789 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:41.432752    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:41.532008 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:41.531940    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:41.799565 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:41.799484    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:42.221689 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:42.221653    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:44.020183 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:44.020152    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:44.120878 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:44.120853    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:44.307309 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:44.307239    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:13:44.340051 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:44.340022    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:44.403235 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:44.403195    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:44.506785 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:44.506758    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:44.866582 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:44.866534    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:44.962555 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:44.962473    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:45.228646 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:45.228618    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:46.221201 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:46.221167    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:46.811019 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:46.810985    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:47.098586 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:47.098491    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:13:59.688335 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:13:59.688300    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:00.789183 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:00.789156    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:14:05.242522 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:05.242482    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:14.593727 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:14.593666    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:14:15.608221 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:15.608192    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:15.673392 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:15.673356    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:15.735569 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:15.735538    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:15.796442 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:15.796412    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:15.851461 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:15.851424    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:15.937111 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:15.937079    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:16.364651 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:16.364554    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:16.664740 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:16.664681    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:17.015491 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:17.015401    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:19.200475 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:19.200415    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:19.291539 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:19.291506    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:20.307351 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.307317    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"]
Sep 07 09:14:20.311084 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.311060    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:20.312979 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.312954    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-dc11d6be-serving-cert\""
Sep 07 09:14:20.321522 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.321491    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"]
Sep 07 09:14:20.336687 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.336664    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:20.359158 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.359134    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13da0c44-af7f-4595-ab79-3858b10acaf3-metrics-certs\") pod \"maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:20.359250 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.359177    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13da0c44-af7f-4595-ab79-3858b10acaf3-openshift-service-ca\") pod \"maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:20.359250 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.359205    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13da0c44-af7f-4595-ab79-3858b10acaf3-maas-api-tls\") pod \"maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:20.359327 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.359255    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj2s5\" (UniqueName: \"kubernetes.io/projected/13da0c44-af7f-4595-ab79-3858b10acaf3-kube-api-access-xj2s5\") pod \"maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:20.391394 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.391362    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:20.460235 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.460210    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13da0c44-af7f-4595-ab79-3858b10acaf3-metrics-certs\") pod \"maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:20.460348 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.460259    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13da0c44-af7f-4595-ab79-3858b10acaf3-openshift-service-ca\") pod \"maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:20.460404 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.460378    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13da0c44-af7f-4595-ab79-3858b10acaf3-maas-api-tls\") pod \"maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:20.460465 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.460442    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj2s5\" (UniqueName: \"kubernetes.io/projected/13da0c44-af7f-4595-ab79-3858b10acaf3-kube-api-access-xj2s5\") pod \"maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:20.460915 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.460897    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13da0c44-af7f-4595-ab79-3858b10acaf3-openshift-service-ca\") pod \"maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:20.462842 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.462818    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13da0c44-af7f-4595-ab79-3858b10acaf3-metrics-certs\") pod \"maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:20.462925 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.462882    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13da0c44-af7f-4595-ab79-3858b10acaf3-maas-api-tls\") pod \"maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:20.467370 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.467349    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj2s5\" (UniqueName: \"kubernetes.io/projected/13da0c44-af7f-4595-ab79-3858b10acaf3-kube-api-access-xj2s5\") pod \"maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:20.494410 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.494383    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:20.622513 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.622433    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:20.814133 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.814102    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:20.896691 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.896615    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"]
Sep 07 09:14:20.898302 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:20.898269    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:20.900092 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:14:20.900067    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13da0c44_af7f_4595_ab79_3858b10acaf3.slice/crio-436e435261cc417ab67f99f0d8057a41964c6393bd3aee3279537cf8786bc51b WatchSource:0}: Error finding container 436e435261cc417ab67f99f0d8057a41964c6393bd3aee3279537cf8786bc51b: Status 404 returned error can't find the container with id 436e435261cc417ab67f99f0d8057a41964c6393bd3aee3279537cf8786bc51b
Sep 07 09:14:21.057489 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:21.057457    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r" event={"ID":"13da0c44-af7f-4595-ab79-3858b10acaf3","Type":"ContainerStarted","Data":"436e435261cc417ab67f99f0d8057a41964c6393bd3aee3279537cf8786bc51b"}
Sep 07 09:14:21.722276 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:21.722238    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:22.064629 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:22.064538    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r" event={"ID":"13da0c44-af7f-4595-ab79-3858b10acaf3","Type":"ContainerStarted","Data":"a75d44bed1d8b969a100f00abbdfdfca8aff330e9427be3d217e335aff9c636c"}
Sep 07 09:14:22.064629 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:22.064608    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:22.079981 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:22.079936    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r" podStartSLOduration=1.6226710949999998 podStartE2EDuration="2.079920197s" totalImagesPullingTime="457.249102ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:14:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:14:22.078106453 +0000 UTC m=+945.042022694" watchObservedRunningTime="2026-09-07 09:14:22.079920197 +0000 UTC m=+945.043836418"
Sep 07 09:14:28.074536 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:28.074511    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:28.295165 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:28.295131    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:14:36.022064 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:36.022020    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:39.000747 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.000698    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"]
Sep 07 09:14:39.004311 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.004287    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.008498 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.008468    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-9d926599-kserve-self-signed-certs\""
Sep 07 09:14:39.016759 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.016733    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"]
Sep 07 09:14:39.036563 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.036535    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:39.134934 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.134877    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l47qz\" (UniqueName: \"kubernetes.io/projected/1650f978-a066-43c9-a7a8-460abc089ebf-kube-api-access-l47qz\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.134934 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.134943    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-dshm\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.135162 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.134986    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1650f978-a066-43c9-a7a8-460abc089ebf-tls-certs\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.135162 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.135064    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-model-cache\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.135162 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.135092    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-tmp-dir\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.135162 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.135146    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-home\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.235778 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.235740    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-model-cache\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.235778 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.235776    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-tmp-dir\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.236016 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.235809    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-home\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.236016 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.235851    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l47qz\" (UniqueName: \"kubernetes.io/projected/1650f978-a066-43c9-a7a8-460abc089ebf-kube-api-access-l47qz\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.236016 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.235876    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-dshm\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.236016 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.235899    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1650f978-a066-43c9-a7a8-460abc089ebf-tls-certs\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.236206 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.236161    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-model-cache\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.236206 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.236190    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-home\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.236280 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.236257    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-tmp-dir\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.238305 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.238281    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-dshm\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.238641 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.238625    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1650f978-a066-43c9-a7a8-460abc089ebf-tls-certs\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.243280 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.243257    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l47qz\" (UniqueName: \"kubernetes.io/projected/1650f978-a066-43c9-a7a8-460abc089ebf-kube-api-access-l47qz\") pod \"e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") " pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.315426 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.315330    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:39.592970 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:39.592895    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"]
Sep 07 09:14:39.596104 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:14:39.596077    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1650f978_a066_43c9_a7a8_460abc089ebf.slice/crio-2351db21e05f8f4918d090072cb3eab403998776873ed2c38ca27cdda1a7ff58 WatchSource:0}: Error finding container 2351db21e05f8f4918d090072cb3eab403998776873ed2c38ca27cdda1a7ff58: Status 404 returned error can't find the container with id 2351db21e05f8f4918d090072cb3eab403998776873ed2c38ca27cdda1a7ff58
Sep 07 09:14:40.158674 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:40.158637    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6" event={"ID":"1650f978-a066-43c9-a7a8-460abc089ebf","Type":"ContainerStarted","Data":"0bae86ac2b5deb9b5b4ffd912e0fb35f4a3e5acf594f4e3d344ad6f03ae28ebe"}
Sep 07 09:14:40.159082 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:40.158685    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6" event={"ID":"1650f978-a066-43c9-a7a8-460abc089ebf","Type":"ContainerStarted","Data":"2351db21e05f8f4918d090072cb3eab403998776873ed2c38ca27cdda1a7ff58"}
Sep 07 09:14:40.175019 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:40.174966    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6" podStartSLOduration=2.174949109 podStartE2EDuration="2.174949109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:14:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:14:40.17416182 +0000 UTC m=+963.138078072" watchObservedRunningTime="2026-09-07 09:14:40.174949109 +0000 UTC m=+963.138865330"
Sep 07 09:14:41.318141 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:41.318107    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:42.800915 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:42.800876    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:14:49.316149 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:49.316115    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:49.316624 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:49.316160    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:49.329186 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:49.329162    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:50.221332 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:50.221305    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:14:50.881951 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:50.881918    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"]
Sep 07 09:14:50.882421 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:50.882162    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r" podUID="13da0c44-af7f-4595-ab79-3858b10acaf3" containerName="maas-api" containerID="cri-o://a75d44bed1d8b969a100f00abbdfdfca8aff330e9427be3d217e335aff9c636c" gracePeriod=30
Sep 07 09:14:50.911618 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:50.911587    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:51.010163 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.010123    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:51.077078 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.077048    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:51.133216 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.133156    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:51.141054 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.141031    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:51.148936 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.148918    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13da0c44-af7f-4595-ab79-3858b10acaf3-maas-api-tls\") pod \"13da0c44-af7f-4595-ab79-3858b10acaf3\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") "
Sep 07 09:14:51.149017 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.148950    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13da0c44-af7f-4595-ab79-3858b10acaf3-metrics-certs\") pod \"13da0c44-af7f-4595-ab79-3858b10acaf3\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") "
Sep 07 09:14:51.149056 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.149031    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13da0c44-af7f-4595-ab79-3858b10acaf3-openshift-service-ca\") pod \"13da0c44-af7f-4595-ab79-3858b10acaf3\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") "
Sep 07 09:14:51.149111 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.149061    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xj2s5\" (UniqueName: \"kubernetes.io/projected/13da0c44-af7f-4595-ab79-3858b10acaf3-kube-api-access-xj2s5\") pod \"13da0c44-af7f-4595-ab79-3858b10acaf3\" (UID: \"13da0c44-af7f-4595-ab79-3858b10acaf3\") "
Sep 07 09:14:51.149371 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.149348    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13da0c44-af7f-4595-ab79-3858b10acaf3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "13da0c44-af7f-4595-ab79-3858b10acaf3" (UID: "13da0c44-af7f-4595-ab79-3858b10acaf3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:14:51.151379 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.151348    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13da0c44-af7f-4595-ab79-3858b10acaf3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "13da0c44-af7f-4595-ab79-3858b10acaf3" (UID: "13da0c44-af7f-4595-ab79-3858b10acaf3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:14:51.151475 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.151383    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13da0c44-af7f-4595-ab79-3858b10acaf3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "13da0c44-af7f-4595-ab79-3858b10acaf3" (UID: "13da0c44-af7f-4595-ab79-3858b10acaf3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:14:51.151475 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.151443    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13da0c44-af7f-4595-ab79-3858b10acaf3-kube-api-access-xj2s5" (OuterVolumeSpecName: "kube-api-access-xj2s5") pod "13da0c44-af7f-4595-ab79-3858b10acaf3" (UID: "13da0c44-af7f-4595-ab79-3858b10acaf3"). InnerVolumeSpecName "kube-api-access-xj2s5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:14:51.218006 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.217978    2574 generic.go:356] "Generic (PLEG): container finished" podID="13da0c44-af7f-4595-ab79-3858b10acaf3" containerID="a75d44bed1d8b969a100f00abbdfdfca8aff330e9427be3d217e335aff9c636c" exitCode=0
Sep 07 09:14:51.218154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.218030    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"
Sep 07 09:14:51.218154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.218067    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r" event={"ID":"13da0c44-af7f-4595-ab79-3858b10acaf3","Type":"ContainerDied","Data":"a75d44bed1d8b969a100f00abbdfdfca8aff330e9427be3d217e335aff9c636c"}
Sep 07 09:14:51.218154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.218117    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r" event={"ID":"13da0c44-af7f-4595-ab79-3858b10acaf3","Type":"ContainerDied","Data":"436e435261cc417ab67f99f0d8057a41964c6393bd3aee3279537cf8786bc51b"}
Sep 07 09:14:51.218154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.218137    2574 scope.go:117] "RemoveContainer" containerID="a75d44bed1d8b969a100f00abbdfdfca8aff330e9427be3d217e335aff9c636c"
Sep 07 09:14:51.227391 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.227374    2574 scope.go:117] "RemoveContainer" containerID="a75d44bed1d8b969a100f00abbdfdfca8aff330e9427be3d217e335aff9c636c"
Sep 07 09:14:51.227671 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:14:51.227654    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a75d44bed1d8b969a100f00abbdfdfca8aff330e9427be3d217e335aff9c636c\": container with ID starting with a75d44bed1d8b969a100f00abbdfdfca8aff330e9427be3d217e335aff9c636c not found: ID does not exist" containerID="a75d44bed1d8b969a100f00abbdfdfca8aff330e9427be3d217e335aff9c636c"
Sep 07 09:14:51.227749 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.227679    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a75d44bed1d8b969a100f00abbdfdfca8aff330e9427be3d217e335aff9c636c"} err="failed to get container status \"a75d44bed1d8b969a100f00abbdfdfca8aff330e9427be3d217e335aff9c636c\": rpc error: code = NotFound desc = could not find container \"a75d44bed1d8b969a100f00abbdfdfca8aff330e9427be3d217e335aff9c636c\": container with ID starting with a75d44bed1d8b969a100f00abbdfdfca8aff330e9427be3d217e335aff9c636c not found: ID does not exist"
Sep 07 09:14:51.237913 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.237892    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"]
Sep 07 09:14:51.245083 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.245062    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-dc11d6be-846bf6d86b-6pt8r"]
Sep 07 09:14:51.249824 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.249807    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13da0c44-af7f-4595-ab79-3858b10acaf3-openshift-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:14:51.249913 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.249825    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xj2s5\" (UniqueName: \"kubernetes.io/projected/13da0c44-af7f-4595-ab79-3858b10acaf3-kube-api-access-xj2s5\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:14:51.249913 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.249836    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13da0c44-af7f-4595-ab79-3858b10acaf3-maas-api-tls\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:14:51.249913 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.249844    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13da0c44-af7f-4595-ab79-3858b10acaf3-metrics-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:14:51.543300 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:51.543268    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13da0c44-af7f-4595-ab79-3858b10acaf3" path="/var/lib/kubelet/pods/13da0c44-af7f-4595-ab79-3858b10acaf3/volumes"
Sep 07 09:14:52.210026 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:52.209999    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:52.599438 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:52.599331    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:52.900097 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:52.900008    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:53.889756 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:53.889722    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:55.387665 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:55.387634    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:55.486905 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:55.486874    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:56.539821 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:56.539780    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:56.597551 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:56.597510    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:57.011116 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:57.011086    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:57.123787 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:57.123751    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:57.804603 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:57.804574    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:14:58.754851 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:58.754761    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:14:58.854239 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:14:58.854209    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:08.601504 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:08.601471    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:15:09.108748 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.108701    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"]
Sep 07 09:15:09.109110 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.109098    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13da0c44-af7f-4595-ab79-3858b10acaf3" containerName="maas-api"
Sep 07 09:15:09.109162 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.109111    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="13da0c44-af7f-4595-ab79-3858b10acaf3" containerName="maas-api"
Sep 07 09:15:09.109197 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.109184    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="13da0c44-af7f-4595-ab79-3858b10acaf3" containerName="maas-api"
Sep 07 09:15:09.115881 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.113867    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.120378 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.120354    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-da4d4c\"/\"default-dockercfg-c7jwd\""
Sep 07 09:15:09.120520 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.120459    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-da4d4c\"/\"autores-model-da4d4c-kserve-self-signed-certs\""
Sep 07 09:15:09.120608 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.120592    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-da4d4c\"/\"kube-root-ca.crt\""
Sep 07 09:15:09.121008 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.120984    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-da4d4c\"/\"openshift-service-ca.crt\""
Sep 07 09:15:09.124457 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.124434    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"]
Sep 07 09:15:09.137135 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.137108    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:09.210109 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.210083    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33e17f3b-90ee-4267-8f37-9f27325f1321-tls-certs\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.210227 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.210122    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm2lj\" (UniqueName: \"kubernetes.io/projected/33e17f3b-90ee-4267-8f37-9f27325f1321-kube-api-access-pm2lj\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.210227 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.210145    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-tmp-dir\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.210386 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.210219    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-dshm\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.210386 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.210263    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-model-cache\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.210386 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.210344    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-home\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.310909 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.310879    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm2lj\" (UniqueName: \"kubernetes.io/projected/33e17f3b-90ee-4267-8f37-9f27325f1321-kube-api-access-pm2lj\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.311036 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.310926    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-tmp-dir\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.311036 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.310956    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-dshm\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.311036 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.310992    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-model-cache\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.311210 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.311055    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-home\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.311210 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.311120    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33e17f3b-90ee-4267-8f37-9f27325f1321-tls-certs\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.311425 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.311405    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-tmp-dir\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.311509 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.311435    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-model-cache\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.311509 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.311478    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-home\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.313543 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.313517    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-dshm\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.313666 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.313645    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33e17f3b-90ee-4267-8f37-9f27325f1321-tls-certs\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.318039 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.318019    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm2lj\" (UniqueName: \"kubernetes.io/projected/33e17f3b-90ee-4267-8f37-9f27325f1321-kube-api-access-pm2lj\") pod \"autores-model-da4d4c-kserve-766f598cc5-8pb9m\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") " pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.430876 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.430853    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:09.702244 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:09.702219    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"]
Sep 07 09:15:09.703833 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:15:09.703802    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33e17f3b_90ee_4267_8f37_9f27325f1321.slice/crio-7e4af4d87c4d6e5a4f4c38b0ed38ceb3c4e1df49c5a1b26d31561ca3d72fd468 WatchSource:0}: Error finding container 7e4af4d87c4d6e5a4f4c38b0ed38ceb3c4e1df49c5a1b26d31561ca3d72fd468: Status 404 returned error can't find the container with id 7e4af4d87c4d6e5a4f4c38b0ed38ceb3c4e1df49c5a1b26d31561ca3d72fd468
Sep 07 09:15:10.319783 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:10.319747    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m" event={"ID":"33e17f3b-90ee-4267-8f37-9f27325f1321","Type":"ContainerStarted","Data":"31540bb18243bdbcc72e25551dffe4c6e399b6c0d0c182bcf3e5ef50418bfed7"}
Sep 07 09:15:10.319953 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:10.319792    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m" event={"ID":"33e17f3b-90ee-4267-8f37-9f27325f1321","Type":"ContainerStarted","Data":"7e4af4d87c4d6e5a4f4c38b0ed38ceb3c4e1df49c5a1b26d31561ca3d72fd468"}
Sep 07 09:15:10.334149 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:10.334093    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m" podStartSLOduration=1.33407583 podStartE2EDuration="1.33407583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:15:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:15:10.333444744 +0000 UTC m=+993.297360983" watchObservedRunningTime="2026-09-07 09:15:10.33407583 +0000 UTC m=+993.297992055"
Sep 07 09:15:19.431731 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:19.431658    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:19.432206 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:19.431760    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:19.443863 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:19.443834    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:20.383077 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:20.383044    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:20.993362 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:20.993332    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"]
Sep 07 09:15:20.997300 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:20.997279    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:20.999974 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:20.999952    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-614a02\"/\"default-dockercfg-tn6sl\""
Sep 07 09:15:21.000108 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:20.999984    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-614a02\"/\"openshift-service-ca.crt\""
Sep 07 09:15:21.000108 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:20.999985    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-614a02\"/\"kube-root-ca.crt\""
Sep 07 09:15:21.000108 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.000053    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-614a02\"/\"test-model-614a02-kserve-self-signed-certs\""
Sep 07 09:15:21.009473 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.009450    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"]
Sep 07 09:15:21.016982 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.016958    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:21.112022 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.111994    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-dshm\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.112172 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.112026    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-home\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.112172 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.112055    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6850039b-9b5d-4a85-b4e8-497abece721f-tls-certs\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.112172 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.112140    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-tmp-dir\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.112344 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.112218    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjn7g\" (UniqueName: \"kubernetes.io/projected/6850039b-9b5d-4a85-b4e8-497abece721f-kube-api-access-cjn7g\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.112344 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.112268    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-model-cache\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.213483 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.213459    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-dshm\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.213624 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.213490    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-home\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.213624 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.213519    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6850039b-9b5d-4a85-b4e8-497abece721f-tls-certs\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.213624 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.213566    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-tmp-dir\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.213624 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.213588    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjn7g\" (UniqueName: \"kubernetes.io/projected/6850039b-9b5d-4a85-b4e8-497abece721f-kube-api-access-cjn7g\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.213624 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.213609    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-model-cache\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.213925 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.213902    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-home\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.214008 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.213987    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-model-cache\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.214069 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.214041    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-tmp-dir\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.216052 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.216031    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-dshm\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.216207 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.216190    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6850039b-9b5d-4a85-b4e8-497abece721f-tls-certs\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.221401 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.221371    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjn7g\" (UniqueName: \"kubernetes.io/projected/6850039b-9b5d-4a85-b4e8-497abece721f-kube-api-access-cjn7g\") pod \"test-model-614a02-kserve-8b7c76649-ktkxd\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") " pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.308496 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.308436    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:21.581545 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.581523    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"]
Sep 07 09:15:21.583357 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:15:21.583332    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6850039b_9b5d_4a85_b4e8_497abece721f.slice/crio-393bd8e9277dc3afcdfdd00b5c3e9bbdba47034b770eb8471f0689087248d1e8 WatchSource:0}: Error finding container 393bd8e9277dc3afcdfdd00b5c3e9bbdba47034b770eb8471f0689087248d1e8: Status 404 returned error can't find the container with id 393bd8e9277dc3afcdfdd00b5c3e9bbdba47034b770eb8471f0689087248d1e8
Sep 07 09:15:21.999111 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:21.999077    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:15:22.385421 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:22.385337    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd" event={"ID":"6850039b-9b5d-4a85-b4e8-497abece721f","Type":"ContainerStarted","Data":"c8528daa4c9a5de31a374cf128720c6ffe70dbd3736ad45f1a281d81815ecfab"}
Sep 07 09:15:22.385421 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:22.385375    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd" event={"ID":"6850039b-9b5d-4a85-b4e8-497abece721f","Type":"ContainerStarted","Data":"393bd8e9277dc3afcdfdd00b5c3e9bbdba47034b770eb8471f0689087248d1e8"}
Sep 07 09:15:22.401327 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:22.401287    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd" podStartSLOduration=2.401273151 podStartE2EDuration="2.401273151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:15:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:15:22.399395637 +0000 UTC m=+1005.363311874" watchObservedRunningTime="2026-09-07 09:15:22.401273151 +0000 UTC m=+1005.365189373"
Sep 07 09:15:22.893663 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:22.893617    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"]
Sep 07 09:15:22.912201 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:22.912171    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:23.375010 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:23.374973    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"]
Sep 07 09:15:23.399976 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:23.399934    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:24.233903 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:24.233874    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"]
Sep 07 09:15:24.255282 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:24.255255    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:24.443416 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:24.443389    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:26.764862 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:26.764813    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:27.261460 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:27.261425    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:27.362966 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:27.362926    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:27.451752 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:27.451701    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:27.508159 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:27.508131    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:27.596126 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:27.596039    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:28.218473 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:28.218436    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:28.285482 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:28.285452    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:28.364080 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:28.364041    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:29.695738 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:29.695672    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:31.309065 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:31.309033    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:31.309428 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:31.309087    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:31.321295 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:31.321271    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:31.443831 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:31.443802    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:15:34.774578 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:34.774541    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:35.175169 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.175134    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"]
Sep 07 09:15:35.175389 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.175345    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4" podUID="43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403" containerName="maas-api" containerID="cri-o://b8da24f234335b8abc6c59d25456c3e2b9d5fe20fc89144710146c4b0b3d28ad" gracePeriod=30
Sep 07 09:15:35.207126 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.207099    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:35.306922 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.306874    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:35.397126 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.397096    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:35.436186 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.436124    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:15:35.453037 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.453012    2574 generic.go:356] "Generic (PLEG): container finished" podID="43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403" containerID="b8da24f234335b8abc6c59d25456c3e2b9d5fe20fc89144710146c4b0b3d28ad" exitCode=0
Sep 07 09:15:35.453160 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.453141    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4" event={"ID":"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403","Type":"ContainerDied","Data":"b8da24f234335b8abc6c59d25456c3e2b9d5fe20fc89144710146c4b0b3d28ad"}
Sep 07 09:15:35.453233 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.453189    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4" event={"ID":"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403","Type":"ContainerDied","Data":"c73e44765f16a8c35bc415efcdb52399cac585ec0dba47343febc521d6051eab"}
Sep 07 09:15:35.453233 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.453216    2574 scope.go:117] "RemoveContainer" containerID="b8da24f234335b8abc6c59d25456c3e2b9d5fe20fc89144710146c4b0b3d28ad"
Sep 07 09:15:35.453373 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.453358    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"
Sep 07 09:15:35.457103 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.457023    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:35.463516 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.463495    2574 scope.go:117] "RemoveContainer" containerID="b8da24f234335b8abc6c59d25456c3e2b9d5fe20fc89144710146c4b0b3d28ad"
Sep 07 09:15:35.463795 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:15:35.463772    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8da24f234335b8abc6c59d25456c3e2b9d5fe20fc89144710146c4b0b3d28ad\": container with ID starting with b8da24f234335b8abc6c59d25456c3e2b9d5fe20fc89144710146c4b0b3d28ad not found: ID does not exist" containerID="b8da24f234335b8abc6c59d25456c3e2b9d5fe20fc89144710146c4b0b3d28ad"
Sep 07 09:15:35.463863 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.463808    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8da24f234335b8abc6c59d25456c3e2b9d5fe20fc89144710146c4b0b3d28ad"} err="failed to get container status \"b8da24f234335b8abc6c59d25456c3e2b9d5fe20fc89144710146c4b0b3d28ad\": rpc error: code = NotFound desc = could not find container \"b8da24f234335b8abc6c59d25456c3e2b9d5fe20fc89144710146c4b0b3d28ad\": container with ID starting with b8da24f234335b8abc6c59d25456c3e2b9d5fe20fc89144710146c4b0b3d28ad not found: ID does not exist"
Sep 07 09:15:35.549006 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.548986    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-metrics-certs\") pod \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") "
Sep 07 09:15:35.549108 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.549075    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4ds8\" (UniqueName: \"kubernetes.io/projected/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-kube-api-access-d4ds8\") pod \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") "
Sep 07 09:15:35.549170 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.549158    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-maas-api-tls\") pod \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") "
Sep 07 09:15:35.549213 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.549188    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-openshift-service-ca\") pod \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\" (UID: \"43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403\") "
Sep 07 09:15:35.549632 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.549602    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403" (UID: "43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:15:35.551283 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.551257    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403" (UID: "43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:15:35.551362 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.551289    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-kube-api-access-d4ds8" (OuterVolumeSpecName: "kube-api-access-d4ds8") pod "43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403" (UID: "43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403"). InnerVolumeSpecName "kube-api-access-d4ds8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:15:35.551506 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.551483    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403" (UID: "43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:15:35.650688 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.650654    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-maas-api-tls\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:35.650688 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.650681    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-openshift-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:35.650912 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.650699    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-metrics-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:35.650912 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.650727    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4ds8\" (UniqueName: \"kubernetes.io/projected/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403-kube-api-access-d4ds8\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:35.727474 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.727385    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:35.778850 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.778825    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"]
Sep 07 09:15:35.783898 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.783875    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-da4d4c-594dc66b67-zhzt4"]
Sep 07 09:15:35.835540 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.835497    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:35.928998 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:35.928971    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:36.029046 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:36.028966    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:36.381277 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:36.381187    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:37.544328 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:37.544299    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403" path="/var/lib/kubelet/pods/43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403/volumes"
Sep 07 09:15:37.916796 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:37.916765    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q" podUID="25292215-7d1f-4930-adcf-85cffc684ef7" containerName="main" containerID="cri-o://1e75f0d65109b0753c0a9a1cdd247f71254089099b710cbf0b6914f483a58e67" gracePeriod=45
Sep 07 09:15:38.176183 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.176122    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:15:38.375098 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.375067    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-model-cache\") pod \"25292215-7d1f-4930-adcf-85cffc684ef7\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") "
Sep 07 09:15:38.375250 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.375130    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-home\") pod \"25292215-7d1f-4930-adcf-85cffc684ef7\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") "
Sep 07 09:15:38.375250 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.375203    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lffmd\" (UniqueName: \"kubernetes.io/projected/25292215-7d1f-4930-adcf-85cffc684ef7-kube-api-access-lffmd\") pod \"25292215-7d1f-4930-adcf-85cffc684ef7\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") "
Sep 07 09:15:38.375250 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.375233    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-dshm\") pod \"25292215-7d1f-4930-adcf-85cffc684ef7\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") "
Sep 07 09:15:38.375394 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.375304    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-tmp-dir\") pod \"25292215-7d1f-4930-adcf-85cffc684ef7\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") "
Sep 07 09:15:38.375394 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.375361    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25292215-7d1f-4930-adcf-85cffc684ef7-tls-certs\") pod \"25292215-7d1f-4930-adcf-85cffc684ef7\" (UID: \"25292215-7d1f-4930-adcf-85cffc684ef7\") "
Sep 07 09:15:38.375486 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.375431    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-model-cache" (OuterVolumeSpecName: "model-cache") pod "25292215-7d1f-4930-adcf-85cffc684ef7" (UID: "25292215-7d1f-4930-adcf-85cffc684ef7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:38.375667 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.375643    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "25292215-7d1f-4930-adcf-85cffc684ef7" (UID: "25292215-7d1f-4930-adcf-85cffc684ef7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:38.375777 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.375658    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-home" (OuterVolumeSpecName: "home") pod "25292215-7d1f-4930-adcf-85cffc684ef7" (UID: "25292215-7d1f-4930-adcf-85cffc684ef7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:38.375777 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.375704    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:38.377608 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.377579    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25292215-7d1f-4930-adcf-85cffc684ef7-kube-api-access-lffmd" (OuterVolumeSpecName: "kube-api-access-lffmd") pod "25292215-7d1f-4930-adcf-85cffc684ef7" (UID: "25292215-7d1f-4930-adcf-85cffc684ef7"). InnerVolumeSpecName "kube-api-access-lffmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:15:38.377608 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.377591    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-dshm" (OuterVolumeSpecName: "dshm") pod "25292215-7d1f-4930-adcf-85cffc684ef7" (UID: "25292215-7d1f-4930-adcf-85cffc684ef7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:38.377836 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.377603    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25292215-7d1f-4930-adcf-85cffc684ef7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "25292215-7d1f-4930-adcf-85cffc684ef7" (UID: "25292215-7d1f-4930-adcf-85cffc684ef7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:15:38.397969 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.397910    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6" podUID="1650f978-a066-43c9-a7a8-460abc089ebf" containerName="main" containerID="cri-o://0bae86ac2b5deb9b5b4ffd912e0fb35f4a3e5acf594f4e3d344ad6f03ae28ebe" gracePeriod=45
Sep 07 09:15:38.400436 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.400409    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:15:38.472024 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.471947    2574 generic.go:356] "Generic (PLEG): container finished" podID="25292215-7d1f-4930-adcf-85cffc684ef7" containerID="1e75f0d65109b0753c0a9a1cdd247f71254089099b710cbf0b6914f483a58e67" exitCode=0
Sep 07 09:15:38.472024 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.471989    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"
Sep 07 09:15:38.472219 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.472025    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q" event={"ID":"25292215-7d1f-4930-adcf-85cffc684ef7","Type":"ContainerDied","Data":"1e75f0d65109b0753c0a9a1cdd247f71254089099b710cbf0b6914f483a58e67"}
Sep 07 09:15:38.472219 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.472065    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q" event={"ID":"25292215-7d1f-4930-adcf-85cffc684ef7","Type":"ContainerDied","Data":"ff858de9db678682c928680a72426c6da7498b2e313965c4c912773fdd0855cd"}
Sep 07 09:15:38.472219 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.472078    2574 scope.go:117] "RemoveContainer" containerID="1e75f0d65109b0753c0a9a1cdd247f71254089099b710cbf0b6914f483a58e67"
Sep 07 09:15:38.476228 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.476211    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:38.476327 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.476230    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lffmd\" (UniqueName: \"kubernetes.io/projected/25292215-7d1f-4930-adcf-85cffc684ef7-kube-api-access-lffmd\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:38.476327 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.476239    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:38.476327 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.476250    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25292215-7d1f-4930-adcf-85cffc684ef7-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:38.476327 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.476257    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25292215-7d1f-4930-adcf-85cffc684ef7-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:38.491914 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.491893    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"]
Sep 07 09:15:38.495955 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.495939    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-9d926599-kserve-6fb65c9974-hgl8q"]
Sep 07 09:15:38.509229 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.509210    2574 scope.go:117] "RemoveContainer" containerID="1e75f0d65109b0753c0a9a1cdd247f71254089099b710cbf0b6914f483a58e67"
Sep 07 09:15:38.509515 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:15:38.509500    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e75f0d65109b0753c0a9a1cdd247f71254089099b710cbf0b6914f483a58e67\": container with ID starting with 1e75f0d65109b0753c0a9a1cdd247f71254089099b710cbf0b6914f483a58e67 not found: ID does not exist" containerID="1e75f0d65109b0753c0a9a1cdd247f71254089099b710cbf0b6914f483a58e67"
Sep 07 09:15:38.509559 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.509523    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e75f0d65109b0753c0a9a1cdd247f71254089099b710cbf0b6914f483a58e67"} err="failed to get container status \"1e75f0d65109b0753c0a9a1cdd247f71254089099b710cbf0b6914f483a58e67\": rpc error: code = NotFound desc = could not find container \"1e75f0d65109b0753c0a9a1cdd247f71254089099b710cbf0b6914f483a58e67\": container with ID starting with 1e75f0d65109b0753c0a9a1cdd247f71254089099b710cbf0b6914f483a58e67 not found: ID does not exist"
Sep 07 09:15:38.637794 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.637771    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:15:38.779565 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.779479    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-home\") pod \"1650f978-a066-43c9-a7a8-460abc089ebf\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") "
Sep 07 09:15:38.779565 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.779540    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l47qz\" (UniqueName: \"kubernetes.io/projected/1650f978-a066-43c9-a7a8-460abc089ebf-kube-api-access-l47qz\") pod \"1650f978-a066-43c9-a7a8-460abc089ebf\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") "
Sep 07 09:15:38.779779 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.779580    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1650f978-a066-43c9-a7a8-460abc089ebf-tls-certs\") pod \"1650f978-a066-43c9-a7a8-460abc089ebf\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") "
Sep 07 09:15:38.779779 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.779615    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-tmp-dir\") pod \"1650f978-a066-43c9-a7a8-460abc089ebf\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") "
Sep 07 09:15:38.779779 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.779654    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-model-cache\") pod \"1650f978-a066-43c9-a7a8-460abc089ebf\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") "
Sep 07 09:15:38.779779 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.779693    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-dshm\") pod \"1650f978-a066-43c9-a7a8-460abc089ebf\" (UID: \"1650f978-a066-43c9-a7a8-460abc089ebf\") "
Sep 07 09:15:38.779914 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.779770    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-home" (OuterVolumeSpecName: "home") pod "1650f978-a066-43c9-a7a8-460abc089ebf" (UID: "1650f978-a066-43c9-a7a8-460abc089ebf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:38.779971 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.779948    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-model-cache" (OuterVolumeSpecName: "model-cache") pod "1650f978-a066-43c9-a7a8-460abc089ebf" (UID: "1650f978-a066-43c9-a7a8-460abc089ebf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:38.780010 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.779955    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1650f978-a066-43c9-a7a8-460abc089ebf" (UID: "1650f978-a066-43c9-a7a8-460abc089ebf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:38.780010 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.779999    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:38.781921 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.781892    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-dshm" (OuterVolumeSpecName: "dshm") pod "1650f978-a066-43c9-a7a8-460abc089ebf" (UID: "1650f978-a066-43c9-a7a8-460abc089ebf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:38.781921 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.781909    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1650f978-a066-43c9-a7a8-460abc089ebf-kube-api-access-l47qz" (OuterVolumeSpecName: "kube-api-access-l47qz") pod "1650f978-a066-43c9-a7a8-460abc089ebf" (UID: "1650f978-a066-43c9-a7a8-460abc089ebf"). InnerVolumeSpecName "kube-api-access-l47qz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:15:38.782075 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.781955    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1650f978-a066-43c9-a7a8-460abc089ebf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1650f978-a066-43c9-a7a8-460abc089ebf" (UID: "1650f978-a066-43c9-a7a8-460abc089ebf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:15:38.880756 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.880697    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:38.880756 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.880750    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:38.880756 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.880760    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1650f978-a066-43c9-a7a8-460abc089ebf-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:38.880977 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.880770    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l47qz\" (UniqueName: \"kubernetes.io/projected/1650f978-a066-43c9-a7a8-460abc089ebf-kube-api-access-l47qz\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:38.880977 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:38.880779    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1650f978-a066-43c9-a7a8-460abc089ebf-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:39.257002 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.256959    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m" podUID="33e17f3b-90ee-4267-8f37-9f27325f1321" containerName="main" containerID="cri-o://31540bb18243bdbcc72e25551dffe4c6e399b6c0d0c182bcf3e5ef50418bfed7" gracePeriod=45
Sep 07 09:15:39.479071 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.479035    2574 generic.go:356] "Generic (PLEG): container finished" podID="1650f978-a066-43c9-a7a8-460abc089ebf" containerID="0bae86ac2b5deb9b5b4ffd912e0fb35f4a3e5acf594f4e3d344ad6f03ae28ebe" exitCode=0
Sep 07 09:15:39.479270 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.479077    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"
Sep 07 09:15:39.479270 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.479073    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6" event={"ID":"1650f978-a066-43c9-a7a8-460abc089ebf","Type":"ContainerDied","Data":"0bae86ac2b5deb9b5b4ffd912e0fb35f4a3e5acf594f4e3d344ad6f03ae28ebe"}
Sep 07 09:15:39.479270 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.479231    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6" event={"ID":"1650f978-a066-43c9-a7a8-460abc089ebf","Type":"ContainerDied","Data":"2351db21e05f8f4918d090072cb3eab403998776873ed2c38ca27cdda1a7ff58"}
Sep 07 09:15:39.479430 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.479267    2574 scope.go:117] "RemoveContainer" containerID="0bae86ac2b5deb9b5b4ffd912e0fb35f4a3e5acf594f4e3d344ad6f03ae28ebe"
Sep 07 09:15:39.482129 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.482106    2574 generic.go:356] "Generic (PLEG): container finished" podID="33e17f3b-90ee-4267-8f37-9f27325f1321" containerID="31540bb18243bdbcc72e25551dffe4c6e399b6c0d0c182bcf3e5ef50418bfed7" exitCode=0
Sep 07 09:15:39.482262 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.482163    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m" event={"ID":"33e17f3b-90ee-4267-8f37-9f27325f1321","Type":"ContainerDied","Data":"31540bb18243bdbcc72e25551dffe4c6e399b6c0d0c182bcf3e5ef50418bfed7"}
Sep 07 09:15:39.489932 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.489911    2574 scope.go:117] "RemoveContainer" containerID="0bae86ac2b5deb9b5b4ffd912e0fb35f4a3e5acf594f4e3d344ad6f03ae28ebe"
Sep 07 09:15:39.490240 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:15:39.490219    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bae86ac2b5deb9b5b4ffd912e0fb35f4a3e5acf594f4e3d344ad6f03ae28ebe\": container with ID starting with 0bae86ac2b5deb9b5b4ffd912e0fb35f4a3e5acf594f4e3d344ad6f03ae28ebe not found: ID does not exist" containerID="0bae86ac2b5deb9b5b4ffd912e0fb35f4a3e5acf594f4e3d344ad6f03ae28ebe"
Sep 07 09:15:39.490318 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.490247    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bae86ac2b5deb9b5b4ffd912e0fb35f4a3e5acf594f4e3d344ad6f03ae28ebe"} err="failed to get container status \"0bae86ac2b5deb9b5b4ffd912e0fb35f4a3e5acf594f4e3d344ad6f03ae28ebe\": rpc error: code = NotFound desc = could not find container \"0bae86ac2b5deb9b5b4ffd912e0fb35f4a3e5acf594f4e3d344ad6f03ae28ebe\": container with ID starting with 0bae86ac2b5deb9b5b4ffd912e0fb35f4a3e5acf594f4e3d344ad6f03ae28ebe not found: ID does not exist"
Sep 07 09:15:39.501357 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.501336    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"]
Sep 07 09:15:39.505578 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.505560    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-9d926599-kserve-7664899f6-rc4h6"]
Sep 07 09:15:39.520360 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.520312    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:39.542784 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.542761    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1650f978-a066-43c9-a7a8-460abc089ebf" path="/var/lib/kubelet/pods/1650f978-a066-43c9-a7a8-460abc089ebf/volumes"
Sep 07 09:15:39.543223 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.543207    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25292215-7d1f-4930-adcf-85cffc684ef7" path="/var/lib/kubelet/pods/25292215-7d1f-4930-adcf-85cffc684ef7/volumes"
Sep 07 09:15:39.584878 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.584858    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33e17f3b-90ee-4267-8f37-9f27325f1321-tls-certs\") pod \"33e17f3b-90ee-4267-8f37-9f27325f1321\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") "
Sep 07 09:15:39.584962 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.584921    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-dshm\") pod \"33e17f3b-90ee-4267-8f37-9f27325f1321\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") "
Sep 07 09:15:39.585049 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.585031    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pm2lj\" (UniqueName: \"kubernetes.io/projected/33e17f3b-90ee-4267-8f37-9f27325f1321-kube-api-access-pm2lj\") pod \"33e17f3b-90ee-4267-8f37-9f27325f1321\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") "
Sep 07 09:15:39.585124 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.585081    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-tmp-dir\") pod \"33e17f3b-90ee-4267-8f37-9f27325f1321\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") "
Sep 07 09:15:39.585124 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.585115    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-home\") pod \"33e17f3b-90ee-4267-8f37-9f27325f1321\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") "
Sep 07 09:15:39.585229 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.585178    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-model-cache\") pod \"33e17f3b-90ee-4267-8f37-9f27325f1321\" (UID: \"33e17f3b-90ee-4267-8f37-9f27325f1321\") "
Sep 07 09:15:39.585336 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.585311    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "33e17f3b-90ee-4267-8f37-9f27325f1321" (UID: "33e17f3b-90ee-4267-8f37-9f27325f1321"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:39.585461 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.585434    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-home" (OuterVolumeSpecName: "home") pod "33e17f3b-90ee-4267-8f37-9f27325f1321" (UID: "33e17f3b-90ee-4267-8f37-9f27325f1321"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:39.585574 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.585467    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-model-cache" (OuterVolumeSpecName: "model-cache") pod "33e17f3b-90ee-4267-8f37-9f27325f1321" (UID: "33e17f3b-90ee-4267-8f37-9f27325f1321"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:39.585700 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.585591    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:39.585700 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.585607    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:39.585700 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.585620    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:39.587406 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.587383    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33e17f3b-90ee-4267-8f37-9f27325f1321-kube-api-access-pm2lj" (OuterVolumeSpecName: "kube-api-access-pm2lj") pod "33e17f3b-90ee-4267-8f37-9f27325f1321" (UID: "33e17f3b-90ee-4267-8f37-9f27325f1321"). InnerVolumeSpecName "kube-api-access-pm2lj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:15:39.587664 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.587636    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33e17f3b-90ee-4267-8f37-9f27325f1321-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "33e17f3b-90ee-4267-8f37-9f27325f1321" (UID: "33e17f3b-90ee-4267-8f37-9f27325f1321"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:15:39.587763 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.587636    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-dshm" (OuterVolumeSpecName: "dshm") pod "33e17f3b-90ee-4267-8f37-9f27325f1321" (UID: "33e17f3b-90ee-4267-8f37-9f27325f1321"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:15:39.686307 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.686280    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33e17f3b-90ee-4267-8f37-9f27325f1321-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:39.686307 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.686304    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33e17f3b-90ee-4267-8f37-9f27325f1321-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:39.686637 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:39.686313    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm2lj\" (UniqueName: \"kubernetes.io/projected/33e17f3b-90ee-4267-8f37-9f27325f1321-kube-api-access-pm2lj\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:15:40.488283 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:40.488194    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"
Sep 07 09:15:40.488283 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:40.488213    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m" event={"ID":"33e17f3b-90ee-4267-8f37-9f27325f1321","Type":"ContainerDied","Data":"7e4af4d87c4d6e5a4f4c38b0ed38ceb3c4e1df49c5a1b26d31561ca3d72fd468"}
Sep 07 09:15:40.488283 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:40.488270    2574 scope.go:117] "RemoveContainer" containerID="31540bb18243bdbcc72e25551dffe4c6e399b6c0d0c182bcf3e5ef50418bfed7"
Sep 07 09:15:40.508470 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:40.508445    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"]
Sep 07 09:15:40.510261 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:40.510236    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-da4d4c/autores-model-da4d4c-kserve-766f598cc5-8pb9m"]
Sep 07 09:15:41.543810 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:41.543769    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33e17f3b-90ee-4267-8f37-9f27325f1321" path="/var/lib/kubelet/pods/33e17f3b-90ee-4267-8f37-9f27325f1321/volumes"
Sep 07 09:15:41.626670 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:41.626630    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:41.729224 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:41.729190    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:41.877255 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:41.877155    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:42.291095 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:42.291050    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:42.624583 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:42.624501    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:43.165785 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:43.165757    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:45.270848 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:45.270813    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:45.375083 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:45.375046    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:46.562080 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:46.562044    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:46.661032 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:46.660994    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:46.945649 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:46.945606    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:47.043975 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:47.043937    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:47.133236 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:47.133202    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:15:52.315907 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:15:52.315874    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:16:02.270219 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:02.270189    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:05.197811 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:05.197778    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:16:08.062237 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:08.062204    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:08.164838 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:08.164808    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:08.261756 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:08.261698    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:10.179771 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.179697    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:10.802910 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.802882    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"]
Sep 07 09:16:10.803298 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.803284    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25292215-7d1f-4930-adcf-85cffc684ef7" containerName="main"
Sep 07 09:16:10.803365 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.803301    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="25292215-7d1f-4930-adcf-85cffc684ef7" containerName="main"
Sep 07 09:16:10.803365 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.803311    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e17f3b-90ee-4267-8f37-9f27325f1321" containerName="main"
Sep 07 09:16:10.803365 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.803316    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e17f3b-90ee-4267-8f37-9f27325f1321" containerName="main"
Sep 07 09:16:10.803365 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.803339    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403" containerName="maas-api"
Sep 07 09:16:10.803365 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.803347    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403" containerName="maas-api"
Sep 07 09:16:10.803365 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.803354    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1650f978-a066-43c9-a7a8-460abc089ebf" containerName="main"
Sep 07 09:16:10.803365 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.803359    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1650f978-a066-43c9-a7a8-460abc089ebf" containerName="main"
Sep 07 09:16:10.803585 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.803422    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1650f978-a066-43c9-a7a8-460abc089ebf" containerName="main"
Sep 07 09:16:10.803585 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.803430    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e17f3b-90ee-4267-8f37-9f27325f1321" containerName="main"
Sep 07 09:16:10.803585 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.803438    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c4bfcc-a77c-4bd5-ac3b-ff00a39c5403" containerName="maas-api"
Sep 07 09:16:10.803585 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.803444    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="25292215-7d1f-4930-adcf-85cffc684ef7" containerName="main"
Sep 07 09:16:10.806723 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.806680    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:10.809470 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.809448    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3bcf58\"/\"openshift-service-ca.crt\""
Sep 07 09:16:10.809663 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.809635    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3bcf58\"/\"test-model-3bcf58-kserve-self-signed-certs\""
Sep 07 09:16:10.809803 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.809514    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-3bcf58\"/\"kube-root-ca.crt\""
Sep 07 09:16:10.809803 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.809593    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-3bcf58\"/\"default-dockercfg-cnf2x\""
Sep 07 09:16:10.816044 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.816019    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"]
Sep 07 09:16:10.824573 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.824537    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:10.970125 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.970093    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-tmp-dir\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:10.970357 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.970131    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-model-cache\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:10.970357 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.970175    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-dshm\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:10.970357 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.970221    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22a6dd85-210d-4f2a-88ad-06731cedeb20-tls-certs\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:10.970357 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.970256    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5lrb\" (UniqueName: \"kubernetes.io/projected/22a6dd85-210d-4f2a-88ad-06731cedeb20-kube-api-access-p5lrb\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:10.970357 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:10.970296    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-home\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:11.070910 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.070831    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-home\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:11.070910 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.070882    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-tmp-dir\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:11.070910 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.070901    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-model-cache\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:11.071315 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.070939    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-dshm\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:11.071315 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.070962    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22a6dd85-210d-4f2a-88ad-06731cedeb20-tls-certs\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:11.071315 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.070995    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5lrb\" (UniqueName: \"kubernetes.io/projected/22a6dd85-210d-4f2a-88ad-06731cedeb20-kube-api-access-p5lrb\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:11.071315 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.071302    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-home\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:11.071529 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.071338    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-tmp-dir\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:11.071529 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.071362    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-model-cache\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:11.073468 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.073450    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-dshm\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:11.073743 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.073706    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22a6dd85-210d-4f2a-88ad-06731cedeb20-tls-certs\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:11.079127 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.079107    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5lrb\" (UniqueName: \"kubernetes.io/projected/22a6dd85-210d-4f2a-88ad-06731cedeb20-kube-api-access-p5lrb\") pod \"test-model-3bcf58-kserve-7684cfdf9-78s6z\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") " pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:11.117937 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.117905    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:11.392166 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.392142    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"]
Sep 07 09:16:11.394314 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:16:11.394283    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22a6dd85_210d_4f2a_88ad_06731cedeb20.slice/crio-594d9276e3e5a446e2bad9604e90de1590ed5498ffad3b4a3219c2387375f222 WatchSource:0}: Error finding container 594d9276e3e5a446e2bad9604e90de1590ed5498ffad3b4a3219c2387375f222: Status 404 returned error can't find the container with id 594d9276e3e5a446e2bad9604e90de1590ed5498ffad3b4a3219c2387375f222
Sep 07 09:16:11.650658 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.650565    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z" event={"ID":"22a6dd85-210d-4f2a-88ad-06731cedeb20","Type":"ContainerStarted","Data":"ad1e2f9c9684ae7684b1914017049c2ec9aac27c6fb777b49746cab6f3a9fb43"}
Sep 07 09:16:11.650658 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.650626    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z" event={"ID":"22a6dd85-210d-4f2a-88ad-06731cedeb20","Type":"ContainerStarted","Data":"594d9276e3e5a446e2bad9604e90de1590ed5498ffad3b4a3219c2387375f222"}
Sep 07 09:16:11.667941 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:11.667900    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z" podStartSLOduration=1.6678833530000001 podStartE2EDuration="1.667883353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:16:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:16:11.666771914 +0000 UTC m=+1054.630688160" watchObservedRunningTime="2026-09-07 09:16:11.667883353 +0000 UTC m=+1054.631799577"
Sep 07 09:16:16.507589 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:16.507550    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:16:17.986932 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:17.986898    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"]
Sep 07 09:16:17.990466 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:17.990448    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:17.992522 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:17.992500    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-085b77\"/\"kube-root-ca.crt\""
Sep 07 09:16:17.992638 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:17.992502    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-085b77\"/\"openshift-service-ca.crt\""
Sep 07 09:16:17.993009 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:17.992995    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-085b77\"/\"default-dockercfg-bsqv8\""
Sep 07 09:16:17.993069 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:17.993031    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-085b77\"/\"explicit-model-085b77-kserve-self-signed-certs\""
Sep 07 09:16:17.998512 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:17.998486    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"]
Sep 07 09:16:18.100022 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.099989    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:18.140340 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.140304    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-dshm\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.140340 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.140340    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-model-cache\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.140588 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.140362    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/367c125c-cb9b-4348-b5ed-40a11159b116-tls-certs\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.140588 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.140443    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-home\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.140588 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.140487    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-tmp-dir\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.140697 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.140627    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftt89\" (UniqueName: \"kubernetes.io/projected/367c125c-cb9b-4348-b5ed-40a11159b116-kube-api-access-ftt89\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.241854 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.241756    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-tmp-dir\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.242057 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.241856    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftt89\" (UniqueName: \"kubernetes.io/projected/367c125c-cb9b-4348-b5ed-40a11159b116-kube-api-access-ftt89\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.242057 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.241894    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-dshm\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.242057 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.241917    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-model-cache\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.242057 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.241942    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/367c125c-cb9b-4348-b5ed-40a11159b116-tls-certs\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.242057 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.242007    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-home\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.242286 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.242268    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-tmp-dir\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.242432 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.242403    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-model-cache\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.242544 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.242472    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-home\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.244496 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.244473    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-dshm\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.244822 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.244802    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/367c125c-cb9b-4348-b5ed-40a11159b116-tls-certs\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.249006 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.248989    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftt89\" (UniqueName: \"kubernetes.io/projected/367c125c-cb9b-4348-b5ed-40a11159b116-kube-api-access-ftt89\") pod \"explicit-model-085b77-kserve-7557b6b667-2vx66\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") " pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.300603 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.300569    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:18.575124 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.575092    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"]
Sep 07 09:16:18.576481 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:16:18.576442    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod367c125c_cb9b_4348_b5ed_40a11159b116.slice/crio-a9bfa5bef40cc0468b7967f29e0e497bafdcf2b90259018f16deed79d9082501 WatchSource:0}: Error finding container a9bfa5bef40cc0468b7967f29e0e497bafdcf2b90259018f16deed79d9082501: Status 404 returned error can't find the container with id a9bfa5bef40cc0468b7967f29e0e497bafdcf2b90259018f16deed79d9082501
Sep 07 09:16:18.687388 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.687356    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66" event={"ID":"367c125c-cb9b-4348-b5ed-40a11159b116","Type":"ContainerStarted","Data":"3ab80abd5f3f7c4216a34aac6d9e0ec7395eec07a689bf41a82f0abcb06efbf7"}
Sep 07 09:16:18.687579 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.687398    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66" event={"ID":"367c125c-cb9b-4348-b5ed-40a11159b116","Type":"ContainerStarted","Data":"a9bfa5bef40cc0468b7967f29e0e497bafdcf2b90259018f16deed79d9082501"}
Sep 07 09:16:18.706874 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:18.706822    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66" podStartSLOduration=1.706804364 podStartE2EDuration="1.706804364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:16:18.705076564 +0000 UTC m=+1061.668992805" watchObservedRunningTime="2026-09-07 09:16:18.706804364 +0000 UTC m=+1061.670720585"
Sep 07 09:16:21.118753 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:21.118689    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:21.118753 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:21.118764    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:21.131451 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:21.131425    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:21.715392 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:21.715362    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:16:24.037289 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:24.037251    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:28.301435 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:28.301393    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:28.302068 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:28.301463    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:28.313746 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:28.313727    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:28.750950 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:28.750916    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:16:29.852480 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:29.852444    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:29.911989 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:29.911956    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:29.991107 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:29.991071    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:32.697395 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:32.697349    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:16:40.521684 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:40.521596    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:41.269352 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:41.269310    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:47.044207 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:47.044170    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:47.143899 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:47.143855    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:47.202926 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:47.202887    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:16:47.241761 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:47.241726    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:47.306303 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:47.306222    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:47.995989 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:47.995949    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:48.079789 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:48.079737    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:49.557500 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:49.557468    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:55.121680 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:55.121644    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"]
Sep 07 09:16:55.143908 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:55.143855    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:55.551284 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:55.551244    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:55.651829 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:55.651796    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:16:56.200692 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:16:56.200655    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:00.099127 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:00.099093    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:17:02.034665 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:02.034634    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:02.117374 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:02.117337    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:02.248424 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:02.248393    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:02.356878 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:02.356795    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:02.444193 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:02.444156    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:02.501331 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:02.501297    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:02.616143 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:02.616035    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:02.698330 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:02.698289    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:02.803945 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:02.803916    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:05.636677 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:05.636640    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:05.733134 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:05.733096    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:06.781596 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:06.781557    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:06.838269 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:06.838236    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:07.157492 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:07.157452    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:07.302127 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:07.302096    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:07.399035 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:07.399005    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:07.880472 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:07.880432    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:10.142078 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.142035    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66" podUID="367c125c-cb9b-4348-b5ed-40a11159b116" containerName="main" containerID="cri-o://3ab80abd5f3f7c4216a34aac6d9e0ec7395eec07a689bf41a82f0abcb06efbf7" gracePeriod=45
Sep 07 09:17:10.401670 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.401616    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:17:10.433012 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.432982    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-model-cache\") pod \"367c125c-cb9b-4348-b5ed-40a11159b116\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") "
Sep 07 09:17:10.433190 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.433024    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-dshm\") pod \"367c125c-cb9b-4348-b5ed-40a11159b116\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") "
Sep 07 09:17:10.433190 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.433051    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-home\") pod \"367c125c-cb9b-4348-b5ed-40a11159b116\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") "
Sep 07 09:17:10.433190 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.433099    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-tmp-dir\") pod \"367c125c-cb9b-4348-b5ed-40a11159b116\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") "
Sep 07 09:17:10.433190 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.433160    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/367c125c-cb9b-4348-b5ed-40a11159b116-tls-certs\") pod \"367c125c-cb9b-4348-b5ed-40a11159b116\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") "
Sep 07 09:17:10.433420 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.433211    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftt89\" (UniqueName: \"kubernetes.io/projected/367c125c-cb9b-4348-b5ed-40a11159b116-kube-api-access-ftt89\") pod \"367c125c-cb9b-4348-b5ed-40a11159b116\" (UID: \"367c125c-cb9b-4348-b5ed-40a11159b116\") "
Sep 07 09:17:10.433420 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.433323    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-model-cache" (OuterVolumeSpecName: "model-cache") pod "367c125c-cb9b-4348-b5ed-40a11159b116" (UID: "367c125c-cb9b-4348-b5ed-40a11159b116"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:10.433420 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.433356    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-home" (OuterVolumeSpecName: "home") pod "367c125c-cb9b-4348-b5ed-40a11159b116" (UID: "367c125c-cb9b-4348-b5ed-40a11159b116"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:10.433529 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.433411    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "367c125c-cb9b-4348-b5ed-40a11159b116" (UID: "367c125c-cb9b-4348-b5ed-40a11159b116"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:10.433629 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.433610    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:17:10.433692 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.433637    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:17:10.433692 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.433652    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:17:10.435508 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.435481    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-dshm" (OuterVolumeSpecName: "dshm") pod "367c125c-cb9b-4348-b5ed-40a11159b116" (UID: "367c125c-cb9b-4348-b5ed-40a11159b116"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:10.435508 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.435499    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/367c125c-cb9b-4348-b5ed-40a11159b116-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "367c125c-cb9b-4348-b5ed-40a11159b116" (UID: "367c125c-cb9b-4348-b5ed-40a11159b116"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:17:10.435644 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.435485    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/367c125c-cb9b-4348-b5ed-40a11159b116-kube-api-access-ftt89" (OuterVolumeSpecName: "kube-api-access-ftt89") pod "367c125c-cb9b-4348-b5ed-40a11159b116" (UID: "367c125c-cb9b-4348-b5ed-40a11159b116"). InnerVolumeSpecName "kube-api-access-ftt89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:17:10.534191 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.534154    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/367c125c-cb9b-4348-b5ed-40a11159b116-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:17:10.534191 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.534183    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/367c125c-cb9b-4348-b5ed-40a11159b116-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:17:10.534191 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.534196    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftt89\" (UniqueName: \"kubernetes.io/projected/367c125c-cb9b-4348-b5ed-40a11159b116-kube-api-access-ftt89\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:17:10.954248 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.954218    2574 generic.go:356] "Generic (PLEG): container finished" podID="367c125c-cb9b-4348-b5ed-40a11159b116" containerID="3ab80abd5f3f7c4216a34aac6d9e0ec7395eec07a689bf41a82f0abcb06efbf7" exitCode=0
Sep 07 09:17:10.954422 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.954257    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"
Sep 07 09:17:10.954422 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.954303    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66" event={"ID":"367c125c-cb9b-4348-b5ed-40a11159b116","Type":"ContainerDied","Data":"3ab80abd5f3f7c4216a34aac6d9e0ec7395eec07a689bf41a82f0abcb06efbf7"}
Sep 07 09:17:10.954422 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.954344    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66" event={"ID":"367c125c-cb9b-4348-b5ed-40a11159b116","Type":"ContainerDied","Data":"a9bfa5bef40cc0468b7967f29e0e497bafdcf2b90259018f16deed79d9082501"}
Sep 07 09:17:10.954422 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.954364    2574 scope.go:117] "RemoveContainer" containerID="3ab80abd5f3f7c4216a34aac6d9e0ec7395eec07a689bf41a82f0abcb06efbf7"
Sep 07 09:17:10.963824 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.963806    2574 scope.go:117] "RemoveContainer" containerID="3ab80abd5f3f7c4216a34aac6d9e0ec7395eec07a689bf41a82f0abcb06efbf7"
Sep 07 09:17:10.964094 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:17:10.964076    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ab80abd5f3f7c4216a34aac6d9e0ec7395eec07a689bf41a82f0abcb06efbf7\": container with ID starting with 3ab80abd5f3f7c4216a34aac6d9e0ec7395eec07a689bf41a82f0abcb06efbf7 not found: ID does not exist" containerID="3ab80abd5f3f7c4216a34aac6d9e0ec7395eec07a689bf41a82f0abcb06efbf7"
Sep 07 09:17:10.964143 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.964101    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ab80abd5f3f7c4216a34aac6d9e0ec7395eec07a689bf41a82f0abcb06efbf7"} err="failed to get container status \"3ab80abd5f3f7c4216a34aac6d9e0ec7395eec07a689bf41a82f0abcb06efbf7\": rpc error: code = NotFound desc = could not find container \"3ab80abd5f3f7c4216a34aac6d9e0ec7395eec07a689bf41a82f0abcb06efbf7\": container with ID starting with 3ab80abd5f3f7c4216a34aac6d9e0ec7395eec07a689bf41a82f0abcb06efbf7 not found: ID does not exist"
Sep 07 09:17:10.973864 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.973670    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"]
Sep 07 09:17:10.975951 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:10.975932    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-085b77/explicit-model-085b77-kserve-7557b6b667-2vx66"]
Sep 07 09:17:11.542540 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:11.542508    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="367c125c-cb9b-4348-b5ed-40a11159b116" path="/var/lib/kubelet/pods/367c125c-cb9b-4348-b5ed-40a11159b116/volumes"
Sep 07 09:17:13.008262 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:13.008234    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:17:13.805255 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:13.805215    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:14.036648 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:14.036615    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:14.489909 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:14.489871    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:14.785820 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:14.785731    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:14.878697 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:14.878657    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:16.148618 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:16.148583    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:16.244289 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:16.244242    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:16.343482 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:16.343441    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:25.427749 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.427698    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"]
Sep 07 09:17:25.428226 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.428090    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="367c125c-cb9b-4348-b5ed-40a11159b116" containerName="main"
Sep 07 09:17:25.428226 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.428101    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="367c125c-cb9b-4348-b5ed-40a11159b116" containerName="main"
Sep 07 09:17:25.428226 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.428176    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="367c125c-cb9b-4348-b5ed-40a11159b116" containerName="main"
Sep 07 09:17:25.431158 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.431139    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.433703 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.433673    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-b0d2d0\"/\"noten-model-b0d2d0-kserve-self-signed-certs\""
Sep 07 09:17:25.433852 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.433740    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-b0d2d0\"/\"openshift-service-ca.crt\""
Sep 07 09:17:25.433852 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.433796    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-b0d2d0\"/\"kube-root-ca.crt\""
Sep 07 09:17:25.433958 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.433857    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-b0d2d0\"/\"default-dockercfg-wjf6q\""
Sep 07 09:17:25.437633 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.437613    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"]
Sep 07 09:17:25.451083 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.451052    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:25.468099 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.468076    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-model-cache\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.468200 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.468107    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-dshm\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.468200 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.468141    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-home\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.468200 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.468173    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98scm\" (UniqueName: \"kubernetes.io/projected/8d312aad-ad65-4c94-8489-6445abeb1e1c-kube-api-access-98scm\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.468200 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.468192    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-tmp-dir\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.468353 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.468212    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d312aad-ad65-4c94-8489-6445abeb1e1c-tls-certs\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.569128 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.569084    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-model-cache\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.569305 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.569283    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-dshm\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.569429 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.569396    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-home\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.569497 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.569458    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98scm\" (UniqueName: \"kubernetes.io/projected/8d312aad-ad65-4c94-8489-6445abeb1e1c-kube-api-access-98scm\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.569497 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.569474    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-model-cache\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.569497 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.569488    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-tmp-dir\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.569761 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.569533    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d312aad-ad65-4c94-8489-6445abeb1e1c-tls-certs\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.569761 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.569641    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-home\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.569884 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.569772    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-tmp-dir\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.571620 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.571598    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-dshm\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.571910 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.571894    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d312aad-ad65-4c94-8489-6445abeb1e1c-tls-certs\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.577415 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.577384    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98scm\" (UniqueName: \"kubernetes.io/projected/8d312aad-ad65-4c94-8489-6445abeb1e1c-kube-api-access-98scm\") pod \"noten-model-b0d2d0-kserve-766d98848c-p76mb\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") " pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:25.741756 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:25.741637    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:26.011785 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:26.011691    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"]
Sep 07 09:17:26.015179 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:17:26.015151    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d312aad_ad65_4c94_8489_6445abeb1e1c.slice/crio-44e769a94f20bd25edcf3659c038f4c82d84abca08b48fff829d76535ba1e4d1 WatchSource:0}: Error finding container 44e769a94f20bd25edcf3659c038f4c82d84abca08b48fff829d76535ba1e4d1: Status 404 returned error can't find the container with id 44e769a94f20bd25edcf3659c038f4c82d84abca08b48fff829d76535ba1e4d1
Sep 07 09:17:26.032450 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:26.032422    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb" event={"ID":"8d312aad-ad65-4c94-8489-6445abeb1e1c","Type":"ContainerStarted","Data":"44e769a94f20bd25edcf3659c038f4c82d84abca08b48fff829d76535ba1e4d1"}
Sep 07 09:17:27.038914 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:27.038881    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb" event={"ID":"8d312aad-ad65-4c94-8489-6445abeb1e1c","Type":"ContainerStarted","Data":"e624295bce96ae72a907975eca8291d39b428bc064f3d31b6e6290ae87cd984a"}
Sep 07 09:17:27.055583 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:27.055534    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb" podStartSLOduration=2.05551783 podStartE2EDuration="2.05551783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:17:27.053910575 +0000 UTC m=+1130.017827006" watchObservedRunningTime="2026-09-07 09:17:27.05551783 +0000 UTC m=+1130.019434051"
Sep 07 09:17:30.560405 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:30.560366    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:31.187339 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:31.187302    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:31.283406 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:31.283372    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:31.604484 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:31.604390    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:35.742497 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:35.742445    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:35.742959 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:35.742790    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:35.755329 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:35.755303    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:36.096231 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:36.096156    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:37.282146 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:37.282105    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:37.342852 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:37.342807    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:39.909085 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:39.909055    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"]
Sep 07 09:17:39.931288 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:39.931253    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:40.046181 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:40.046146    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:40.249447 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:40.249415    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:40.351586 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:40.351545    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:45.432632 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:45.432574    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:48.118877 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:48.118839    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:48.219877 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:48.219845    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:48.279973 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:48.279936    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:48.342919 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:48.342861    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:48.648441 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:48.648405    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:49.682139 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:49.682101    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"]
Sep 07 09:17:49.702287 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:49.702254    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:50.705270 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:50.705243    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"]
Sep 07 09:17:50.727514 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:50.727482    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:50.921027 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:50.920992    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:54.104410 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:54.104373    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:17:54.524052 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:54.524017    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:54.624222 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:54.624183    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:54.787448 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:54.787352    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:54.929970 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:54.929919    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb" podUID="8d312aad-ad65-4c94-8489-6445abeb1e1c" containerName="main" containerID="cri-o://e624295bce96ae72a907975eca8291d39b428bc064f3d31b6e6290ae87cd984a" gracePeriod=45
Sep 07 09:17:55.129502 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.129467    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:55.188216 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.188188    2574 generic.go:356] "Generic (PLEG): container finished" podID="8d312aad-ad65-4c94-8489-6445abeb1e1c" containerID="e624295bce96ae72a907975eca8291d39b428bc064f3d31b6e6290ae87cd984a" exitCode=0
Sep 07 09:17:55.188353 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.188263    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb" event={"ID":"8d312aad-ad65-4c94-8489-6445abeb1e1c","Type":"ContainerDied","Data":"e624295bce96ae72a907975eca8291d39b428bc064f3d31b6e6290ae87cd984a"}
Sep 07 09:17:55.188353 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.188309    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb" event={"ID":"8d312aad-ad65-4c94-8489-6445abeb1e1c","Type":"ContainerDied","Data":"44e769a94f20bd25edcf3659c038f4c82d84abca08b48fff829d76535ba1e4d1"}
Sep 07 09:17:55.188353 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.188325    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44e769a94f20bd25edcf3659c038f4c82d84abca08b48fff829d76535ba1e4d1"
Sep 07 09:17:55.192593 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.192576    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:55.212697 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.212668    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:55.263957 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.263925    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-dshm\") pod \"8d312aad-ad65-4c94-8489-6445abeb1e1c\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") "
Sep 07 09:17:55.264140 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.263984    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d312aad-ad65-4c94-8489-6445abeb1e1c-tls-certs\") pod \"8d312aad-ad65-4c94-8489-6445abeb1e1c\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") "
Sep 07 09:17:55.264140 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.264018    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-home\") pod \"8d312aad-ad65-4c94-8489-6445abeb1e1c\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") "
Sep 07 09:17:55.264140 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.264124    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98scm\" (UniqueName: \"kubernetes.io/projected/8d312aad-ad65-4c94-8489-6445abeb1e1c-kube-api-access-98scm\") pod \"8d312aad-ad65-4c94-8489-6445abeb1e1c\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") "
Sep 07 09:17:55.264322 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.264156    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-tmp-dir\") pod \"8d312aad-ad65-4c94-8489-6445abeb1e1c\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") "
Sep 07 09:17:55.264322 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.264211    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-home" (OuterVolumeSpecName: "home") pod "8d312aad-ad65-4c94-8489-6445abeb1e1c" (UID: "8d312aad-ad65-4c94-8489-6445abeb1e1c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:55.264322 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.264217    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-model-cache\") pod \"8d312aad-ad65-4c94-8489-6445abeb1e1c\" (UID: \"8d312aad-ad65-4c94-8489-6445abeb1e1c\") "
Sep 07 09:17:55.264484 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.264432    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-model-cache" (OuterVolumeSpecName: "model-cache") pod "8d312aad-ad65-4c94-8489-6445abeb1e1c" (UID: "8d312aad-ad65-4c94-8489-6445abeb1e1c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:55.264539 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.264470    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8d312aad-ad65-4c94-8489-6445abeb1e1c" (UID: "8d312aad-ad65-4c94-8489-6445abeb1e1c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:55.264620 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.264604    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:17:55.264881 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.264620    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:17:55.264881 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.264629    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:17:55.266347 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.266323    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-dshm" (OuterVolumeSpecName: "dshm") pod "8d312aad-ad65-4c94-8489-6445abeb1e1c" (UID: "8d312aad-ad65-4c94-8489-6445abeb1e1c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:17:55.266449 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.266390    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d312aad-ad65-4c94-8489-6445abeb1e1c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8d312aad-ad65-4c94-8489-6445abeb1e1c" (UID: "8d312aad-ad65-4c94-8489-6445abeb1e1c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:17:55.266449 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.266392    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d312aad-ad65-4c94-8489-6445abeb1e1c-kube-api-access-98scm" (OuterVolumeSpecName: "kube-api-access-98scm") pod "8d312aad-ad65-4c94-8489-6445abeb1e1c" (UID: "8d312aad-ad65-4c94-8489-6445abeb1e1c"). InnerVolumeSpecName "kube-api-access-98scm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:17:55.354238 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.354153    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:17:55.365331 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.365299    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d312aad-ad65-4c94-8489-6445abeb1e1c-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:17:55.365331 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.365326    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d312aad-ad65-4c94-8489-6445abeb1e1c-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:17:55.365509 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:55.365341    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98scm\" (UniqueName: \"kubernetes.io/projected/8d312aad-ad65-4c94-8489-6445abeb1e1c-kube-api-access-98scm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:17:56.193535 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:56.193500    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"
Sep 07 09:17:56.207513 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:56.207489    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"]
Sep 07 09:17:56.211653 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:56.211630    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-b0d2d0/noten-model-b0d2d0-kserve-766d98848c-p76mb"]
Sep 07 09:17:57.543202 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:17:57.543166    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d312aad-ad65-4c94-8489-6445abeb1e1c" path="/var/lib/kubelet/pods/8d312aad-ad65-4c94-8489-6445abeb1e1c/volumes"
Sep 07 09:18:00.967896 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:00.967812    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:01.071513 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:01.071475    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:01.166798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:01.166762    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:01.323199 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:01.323102    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:01.422925 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:01.422898    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:01.512254 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:01.512223    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:01.581791 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:01.581675    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:01.928175 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:01.928139    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:03.291501 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:03.291459    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:18:04.704463 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:04.704424    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd" podUID="6850039b-9b5d-4a85-b4e8-497abece721f" containerName="main" containerID="cri-o://c8528daa4c9a5de31a374cf128720c6ffe70dbd3736ad45f1a281d81815ecfab" gracePeriod=45
Sep 07 09:18:04.960809 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:04.960747    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:18:05.053078 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.053044    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-tmp-dir\") pod \"6850039b-9b5d-4a85-b4e8-497abece721f\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") "
Sep 07 09:18:05.053078 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.053092    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-model-cache\") pod \"6850039b-9b5d-4a85-b4e8-497abece721f\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") "
Sep 07 09:18:05.053357 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.053119    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-dshm\") pod \"6850039b-9b5d-4a85-b4e8-497abece721f\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") "
Sep 07 09:18:05.053357 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.053160    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-home\") pod \"6850039b-9b5d-4a85-b4e8-497abece721f\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") "
Sep 07 09:18:05.053357 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.053182    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6850039b-9b5d-4a85-b4e8-497abece721f-tls-certs\") pod \"6850039b-9b5d-4a85-b4e8-497abece721f\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") "
Sep 07 09:18:05.053357 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.053209    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjn7g\" (UniqueName: \"kubernetes.io/projected/6850039b-9b5d-4a85-b4e8-497abece721f-kube-api-access-cjn7g\") pod \"6850039b-9b5d-4a85-b4e8-497abece721f\" (UID: \"6850039b-9b5d-4a85-b4e8-497abece721f\") "
Sep 07 09:18:05.053550 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.053370    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6850039b-9b5d-4a85-b4e8-497abece721f" (UID: "6850039b-9b5d-4a85-b4e8-497abece721f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:05.053550 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.053442    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-model-cache" (OuterVolumeSpecName: "model-cache") pod "6850039b-9b5d-4a85-b4e8-497abece721f" (UID: "6850039b-9b5d-4a85-b4e8-497abece721f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:05.053550 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.053509    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-home" (OuterVolumeSpecName: "home") pod "6850039b-9b5d-4a85-b4e8-497abece721f" (UID: "6850039b-9b5d-4a85-b4e8-497abece721f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:05.053663 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.053630    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:05.053663 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.053649    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:05.053858 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.053664    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:05.055734 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.055684    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6850039b-9b5d-4a85-b4e8-497abece721f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6850039b-9b5d-4a85-b4e8-497abece721f" (UID: "6850039b-9b5d-4a85-b4e8-497abece721f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:18:05.055839 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.055699    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6850039b-9b5d-4a85-b4e8-497abece721f-kube-api-access-cjn7g" (OuterVolumeSpecName: "kube-api-access-cjn7g") pod "6850039b-9b5d-4a85-b4e8-497abece721f" (UID: "6850039b-9b5d-4a85-b4e8-497abece721f"). InnerVolumeSpecName "kube-api-access-cjn7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:18:05.055839 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.055770    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-dshm" (OuterVolumeSpecName: "dshm") pod "6850039b-9b5d-4a85-b4e8-497abece721f" (UID: "6850039b-9b5d-4a85-b4e8-497abece721f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:05.155011 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.154971    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6850039b-9b5d-4a85-b4e8-497abece721f-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:05.155011 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.155004    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6850039b-9b5d-4a85-b4e8-497abece721f-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:05.155225 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.155028    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjn7g\" (UniqueName: \"kubernetes.io/projected/6850039b-9b5d-4a85-b4e8-497abece721f-kube-api-access-cjn7g\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:05.248570 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.248470    2574 generic.go:356] "Generic (PLEG): container finished" podID="6850039b-9b5d-4a85-b4e8-497abece721f" containerID="c8528daa4c9a5de31a374cf128720c6ffe70dbd3736ad45f1a281d81815ecfab" exitCode=0
Sep 07 09:18:05.248570 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.248525    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"
Sep 07 09:18:05.248570 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.248555    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd" event={"ID":"6850039b-9b5d-4a85-b4e8-497abece721f","Type":"ContainerDied","Data":"c8528daa4c9a5de31a374cf128720c6ffe70dbd3736ad45f1a281d81815ecfab"}
Sep 07 09:18:05.248885 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.248597    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd" event={"ID":"6850039b-9b5d-4a85-b4e8-497abece721f","Type":"ContainerDied","Data":"393bd8e9277dc3afcdfdd00b5c3e9bbdba47034b770eb8471f0689087248d1e8"}
Sep 07 09:18:05.248885 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.248615    2574 scope.go:117] "RemoveContainer" containerID="c8528daa4c9a5de31a374cf128720c6ffe70dbd3736ad45f1a281d81815ecfab"
Sep 07 09:18:05.258280 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.258261    2574 scope.go:117] "RemoveContainer" containerID="c8528daa4c9a5de31a374cf128720c6ffe70dbd3736ad45f1a281d81815ecfab"
Sep 07 09:18:05.258552 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:18:05.258530    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8528daa4c9a5de31a374cf128720c6ffe70dbd3736ad45f1a281d81815ecfab\": container with ID starting with c8528daa4c9a5de31a374cf128720c6ffe70dbd3736ad45f1a281d81815ecfab not found: ID does not exist" containerID="c8528daa4c9a5de31a374cf128720c6ffe70dbd3736ad45f1a281d81815ecfab"
Sep 07 09:18:05.258664 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.258557    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8528daa4c9a5de31a374cf128720c6ffe70dbd3736ad45f1a281d81815ecfab"} err="failed to get container status \"c8528daa4c9a5de31a374cf128720c6ffe70dbd3736ad45f1a281d81815ecfab\": rpc error: code = NotFound desc = could not find container \"c8528daa4c9a5de31a374cf128720c6ffe70dbd3736ad45f1a281d81815ecfab\": container with ID starting with c8528daa4c9a5de31a374cf128720c6ffe70dbd3736ad45f1a281d81815ecfab not found: ID does not exist"
Sep 07 09:18:05.268969 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.268947    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"]
Sep 07 09:18:05.274552 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.274532    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-614a02/test-model-614a02-kserve-8b7c76649-ktkxd"]
Sep 07 09:18:05.542991 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.542911    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6850039b-9b5d-4a85-b4e8-497abece721f" path="/var/lib/kubelet/pods/6850039b-9b5d-4a85-b4e8-497abece721f/volumes"
Sep 07 09:18:05.725970 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.725933    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z" podUID="22a6dd85-210d-4f2a-88ad-06731cedeb20" containerName="main" containerID="cri-o://ad1e2f9c9684ae7684b1914017049c2ec9aac27c6fb777b49746cab6f3a9fb43" gracePeriod=45
Sep 07 09:18:05.976833 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:05.976812    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:18:06.063840 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.063809    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5lrb\" (UniqueName: \"kubernetes.io/projected/22a6dd85-210d-4f2a-88ad-06731cedeb20-kube-api-access-p5lrb\") pod \"22a6dd85-210d-4f2a-88ad-06731cedeb20\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") "
Sep 07 09:18:06.063997 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.063846    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-dshm\") pod \"22a6dd85-210d-4f2a-88ad-06731cedeb20\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") "
Sep 07 09:18:06.063997 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.063876    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22a6dd85-210d-4f2a-88ad-06731cedeb20-tls-certs\") pod \"22a6dd85-210d-4f2a-88ad-06731cedeb20\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") "
Sep 07 09:18:06.063997 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.063903    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-tmp-dir\") pod \"22a6dd85-210d-4f2a-88ad-06731cedeb20\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") "
Sep 07 09:18:06.063997 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.063980    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-home\") pod \"22a6dd85-210d-4f2a-88ad-06731cedeb20\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") "
Sep 07 09:18:06.064206 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.064018    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-model-cache\") pod \"22a6dd85-210d-4f2a-88ad-06731cedeb20\" (UID: \"22a6dd85-210d-4f2a-88ad-06731cedeb20\") "
Sep 07 09:18:06.064295 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.064270    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "22a6dd85-210d-4f2a-88ad-06731cedeb20" (UID: "22a6dd85-210d-4f2a-88ad-06731cedeb20"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:06.064363 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.064283    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-home" (OuterVolumeSpecName: "home") pod "22a6dd85-210d-4f2a-88ad-06731cedeb20" (UID: "22a6dd85-210d-4f2a-88ad-06731cedeb20"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:06.064405 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.064359    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-model-cache" (OuterVolumeSpecName: "model-cache") pod "22a6dd85-210d-4f2a-88ad-06731cedeb20" (UID: "22a6dd85-210d-4f2a-88ad-06731cedeb20"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:06.066327 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.066297    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-dshm" (OuterVolumeSpecName: "dshm") pod "22a6dd85-210d-4f2a-88ad-06731cedeb20" (UID: "22a6dd85-210d-4f2a-88ad-06731cedeb20"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:18:06.066441 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.066393    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22a6dd85-210d-4f2a-88ad-06731cedeb20-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "22a6dd85-210d-4f2a-88ad-06731cedeb20" (UID: "22a6dd85-210d-4f2a-88ad-06731cedeb20"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:18:06.066441 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.066412    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22a6dd85-210d-4f2a-88ad-06731cedeb20-kube-api-access-p5lrb" (OuterVolumeSpecName: "kube-api-access-p5lrb") pod "22a6dd85-210d-4f2a-88ad-06731cedeb20" (UID: "22a6dd85-210d-4f2a-88ad-06731cedeb20"). InnerVolumeSpecName "kube-api-access-p5lrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:18:06.164857 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.164821    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:06.164857 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.164853    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:06.165055 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.164869    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5lrb\" (UniqueName: \"kubernetes.io/projected/22a6dd85-210d-4f2a-88ad-06731cedeb20-kube-api-access-p5lrb\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:06.165055 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.164880    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:06.165055 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.164892    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22a6dd85-210d-4f2a-88ad-06731cedeb20-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:06.165055 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.164902    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22a6dd85-210d-4f2a-88ad-06731cedeb20-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:06.255902 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.255865    2574 generic.go:356] "Generic (PLEG): container finished" podID="22a6dd85-210d-4f2a-88ad-06731cedeb20" containerID="ad1e2f9c9684ae7684b1914017049c2ec9aac27c6fb777b49746cab6f3a9fb43" exitCode=0
Sep 07 09:18:06.256080 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.255917    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"
Sep 07 09:18:06.256080 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.255951    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z" event={"ID":"22a6dd85-210d-4f2a-88ad-06731cedeb20","Type":"ContainerDied","Data":"ad1e2f9c9684ae7684b1914017049c2ec9aac27c6fb777b49746cab6f3a9fb43"}
Sep 07 09:18:06.256080 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.256010    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z" event={"ID":"22a6dd85-210d-4f2a-88ad-06731cedeb20","Type":"ContainerDied","Data":"594d9276e3e5a446e2bad9604e90de1590ed5498ffad3b4a3219c2387375f222"}
Sep 07 09:18:06.256080 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.256037    2574 scope.go:117] "RemoveContainer" containerID="ad1e2f9c9684ae7684b1914017049c2ec9aac27c6fb777b49746cab6f3a9fb43"
Sep 07 09:18:06.265193 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.265174    2574 scope.go:117] "RemoveContainer" containerID="ad1e2f9c9684ae7684b1914017049c2ec9aac27c6fb777b49746cab6f3a9fb43"
Sep 07 09:18:06.265484 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:18:06.265463    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad1e2f9c9684ae7684b1914017049c2ec9aac27c6fb777b49746cab6f3a9fb43\": container with ID starting with ad1e2f9c9684ae7684b1914017049c2ec9aac27c6fb777b49746cab6f3a9fb43 not found: ID does not exist" containerID="ad1e2f9c9684ae7684b1914017049c2ec9aac27c6fb777b49746cab6f3a9fb43"
Sep 07 09:18:06.265561 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.265491    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad1e2f9c9684ae7684b1914017049c2ec9aac27c6fb777b49746cab6f3a9fb43"} err="failed to get container status \"ad1e2f9c9684ae7684b1914017049c2ec9aac27c6fb777b49746cab6f3a9fb43\": rpc error: code = NotFound desc = could not find container \"ad1e2f9c9684ae7684b1914017049c2ec9aac27c6fb777b49746cab6f3a9fb43\": container with ID starting with ad1e2f9c9684ae7684b1914017049c2ec9aac27c6fb777b49746cab6f3a9fb43 not found: ID does not exist"
Sep 07 09:18:06.275897 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.275874    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"]
Sep 07 09:18:06.281703 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:06.281683    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-3bcf58/test-model-3bcf58-kserve-7684cfdf9-78s6z"]
Sep 07 09:18:07.507418 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:07.507390    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:07.542510 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:07.542472    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22a6dd85-210d-4f2a-88ad-06731cedeb20" path="/var/lib/kubelet/pods/22a6dd85-210d-4f2a-88ad-06731cedeb20/volumes"
Sep 07 09:18:07.566078 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:07.566052    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:08.008033 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:08.007999    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:08.067633 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:08.067600    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:08.256298 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:08.256255    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:08.408863 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:08.408815    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:08.721233 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:08.721148    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:08.826657 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:08.826615    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:09.310699 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:09.310659    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:12.049561 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:12.049522    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:12.148688 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:12.148652    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:13.346584 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:13.346546    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:13.446472 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:13.446444    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:13.775405 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:13.775372    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:13.876056 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:13.876016    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:13.977550 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:13.977513    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:17.993475 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:17.993438    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:18:18.736496 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:18.736454    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:19.317705 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.317672    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"]
Sep 07 09:18:19.318235 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.317917    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5" podUID="2d8be2de-450b-46df-a2cf-37cdf62c02db" containerName="maas-api" containerID="cri-o://39519caa8e526d9a04f5672791e1f321cced92ae6599eb0e4b1446cc016db0e7" gracePeriod=30
Sep 07 09:18:19.348909 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.348879    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:19.412306 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.412273    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:19.467891 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.467806    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:19.527511 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.527474    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:19.573256 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.573205    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:18:19.582447 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.582421    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:19.688432 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.688406    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-metrics-certs\") pod \"2d8be2de-450b-46df-a2cf-37cdf62c02db\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") "
Sep 07 09:18:19.688624 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.688446    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d8be2de-450b-46df-a2cf-37cdf62c02db-openshift-service-ca\") pod \"2d8be2de-450b-46df-a2cf-37cdf62c02db\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") "
Sep 07 09:18:19.688624 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.688487    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6qj7\" (UniqueName: \"kubernetes.io/projected/2d8be2de-450b-46df-a2cf-37cdf62c02db-kube-api-access-m6qj7\") pod \"2d8be2de-450b-46df-a2cf-37cdf62c02db\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") "
Sep 07 09:18:19.688624 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.688616    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-maas-api-tls\") pod \"2d8be2de-450b-46df-a2cf-37cdf62c02db\" (UID: \"2d8be2de-450b-46df-a2cf-37cdf62c02db\") "
Sep 07 09:18:19.688916 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.688880    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d8be2de-450b-46df-a2cf-37cdf62c02db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2d8be2de-450b-46df-a2cf-37cdf62c02db" (UID: "2d8be2de-450b-46df-a2cf-37cdf62c02db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:18:19.689038 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.689017    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d8be2de-450b-46df-a2cf-37cdf62c02db-openshift-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:19.690833 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.690809    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2d8be2de-450b-46df-a2cf-37cdf62c02db" (UID: "2d8be2de-450b-46df-a2cf-37cdf62c02db"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:18:19.690936 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.690847    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d8be2de-450b-46df-a2cf-37cdf62c02db-kube-api-access-m6qj7" (OuterVolumeSpecName: "kube-api-access-m6qj7") pod "2d8be2de-450b-46df-a2cf-37cdf62c02db" (UID: "2d8be2de-450b-46df-a2cf-37cdf62c02db"). InnerVolumeSpecName "kube-api-access-m6qj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:18:19.690936 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.690891    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2d8be2de-450b-46df-a2cf-37cdf62c02db" (UID: "2d8be2de-450b-46df-a2cf-37cdf62c02db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:18:19.790456 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.790415    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6qj7\" (UniqueName: \"kubernetes.io/projected/2d8be2de-450b-46df-a2cf-37cdf62c02db-kube-api-access-m6qj7\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:19.790456 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.790446    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-maas-api-tls\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:19.790456 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.790459    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d8be2de-450b-46df-a2cf-37cdf62c02db-metrics-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:18:19.902524 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:19.902430    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:20.338769 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:20.338704    2574 generic.go:356] "Generic (PLEG): container finished" podID="2d8be2de-450b-46df-a2cf-37cdf62c02db" containerID="39519caa8e526d9a04f5672791e1f321cced92ae6599eb0e4b1446cc016db0e7" exitCode=0
Sep 07 09:18:20.339244 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:20.338778    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"
Sep 07 09:18:20.339244 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:20.338809    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5" event={"ID":"2d8be2de-450b-46df-a2cf-37cdf62c02db","Type":"ContainerDied","Data":"39519caa8e526d9a04f5672791e1f321cced92ae6599eb0e4b1446cc016db0e7"}
Sep 07 09:18:20.339244 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:20.338899    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5" event={"ID":"2d8be2de-450b-46df-a2cf-37cdf62c02db","Type":"ContainerDied","Data":"35dc873b3ccec67182b4d2097be9a1ff399eacb243e10e153794e58432f029b1"}
Sep 07 09:18:20.339244 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:20.338928    2574 scope.go:117] "RemoveContainer" containerID="39519caa8e526d9a04f5672791e1f321cced92ae6599eb0e4b1446cc016db0e7"
Sep 07 09:18:20.349360 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:20.349344    2574 scope.go:117] "RemoveContainer" containerID="39519caa8e526d9a04f5672791e1f321cced92ae6599eb0e4b1446cc016db0e7"
Sep 07 09:18:20.349697 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:18:20.349676    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39519caa8e526d9a04f5672791e1f321cced92ae6599eb0e4b1446cc016db0e7\": container with ID starting with 39519caa8e526d9a04f5672791e1f321cced92ae6599eb0e4b1446cc016db0e7 not found: ID does not exist" containerID="39519caa8e526d9a04f5672791e1f321cced92ae6599eb0e4b1446cc016db0e7"
Sep 07 09:18:20.349846 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:20.349703    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39519caa8e526d9a04f5672791e1f321cced92ae6599eb0e4b1446cc016db0e7"} err="failed to get container status \"39519caa8e526d9a04f5672791e1f321cced92ae6599eb0e4b1446cc016db0e7\": rpc error: code = NotFound desc = could not find container \"39519caa8e526d9a04f5672791e1f321cced92ae6599eb0e4b1446cc016db0e7\": container with ID starting with 39519caa8e526d9a04f5672791e1f321cced92ae6599eb0e4b1446cc016db0e7 not found: ID does not exist"
Sep 07 09:18:20.362160 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:20.362134    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"]
Sep 07 09:18:20.365525 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:20.365502    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3bcf58-5f6d9c7d97-wz9k5"]
Sep 07 09:18:21.541954 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:21.541913    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d8be2de-450b-46df-a2cf-37cdf62c02db" path="/var/lib/kubelet/pods/2d8be2de-450b-46df-a2cf-37cdf62c02db/volumes"
Sep 07 09:18:22.031042 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:22.031015    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:25.707656 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:25.707625    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:25.761973 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:25.761942    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:25.959964 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:25.959880    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:26.520128 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:26.520088    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:26.815469 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:26.815389    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:27.027453 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:27.027423    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:27.604169 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:27.604137    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:18:30.170366 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:30.170328    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:30.259935 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:30.259904    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:31.283589 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.283558    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"]
Sep 07 09:18:31.284130 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.284109    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22a6dd85-210d-4f2a-88ad-06731cedeb20" containerName="main"
Sep 07 09:18:31.284233 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.284132    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="22a6dd85-210d-4f2a-88ad-06731cedeb20" containerName="main"
Sep 07 09:18:31.284233 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.284154    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6850039b-9b5d-4a85-b4e8-497abece721f" containerName="main"
Sep 07 09:18:31.284233 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.284163    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6850039b-9b5d-4a85-b4e8-497abece721f" containerName="main"
Sep 07 09:18:31.284233 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.284174    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d8be2de-450b-46df-a2cf-37cdf62c02db" containerName="maas-api"
Sep 07 09:18:31.284233 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.284182    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d8be2de-450b-46df-a2cf-37cdf62c02db" containerName="maas-api"
Sep 07 09:18:31.284233 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.284193    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d312aad-ad65-4c94-8489-6445abeb1e1c" containerName="main"
Sep 07 09:18:31.284233 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.284202    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d312aad-ad65-4c94-8489-6445abeb1e1c" containerName="main"
Sep 07 09:18:31.284557 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.284323    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d8be2de-450b-46df-a2cf-37cdf62c02db" containerName="maas-api"
Sep 07 09:18:31.284557 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.284335    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="22a6dd85-210d-4f2a-88ad-06731cedeb20" containerName="main"
Sep 07 09:18:31.284557 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.284344    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d312aad-ad65-4c94-8489-6445abeb1e1c" containerName="main"
Sep 07 09:18:31.284557 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.284355    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6850039b-9b5d-4a85-b4e8-497abece721f" containerName="main"
Sep 07 09:18:31.289250 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.289225    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:31.291874 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.291683    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 09:18:31.291874 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.291667    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 07 09:18:31.291874 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.291730    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 07 09:18:31.291874 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.291771    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rckvc\""
Sep 07 09:18:31.291874 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.291686    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-3dc031-serving-cert\""
Sep 07 09:18:31.295625 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.295568    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"]
Sep 07 09:18:31.328271 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.328247    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:31.393724 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.393689    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:31.393890 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.393746    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mvkw\" (UniqueName: \"kubernetes.io/projected/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-kube-api-access-4mvkw\") pod \"maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:31.393890 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.393857    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:31.393974 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.393907    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:31.449998 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.449961    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:31.495094 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.495065    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mvkw\" (UniqueName: \"kubernetes.io/projected/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-kube-api-access-4mvkw\") pod \"maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:31.495254 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.495126    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:31.495254 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.495164    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:31.495254 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.495183    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:31.495428 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:18:31.495289    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-serving-cert: secret "maas-api-e2e-shared-a-w3-3dc031-serving-cert" not found
Sep 07 09:18:31.495428 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:18:31.495371    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-maas-api-tls podName:ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa nodeName:}" failed. No retries permitted until 2026-09-07 09:18:31.995350387 +0000 UTC m=+1194.959266589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-maas-api-tls") pod "maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk" (UID: "ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa") : secret "maas-api-e2e-shared-a-w3-3dc031-serving-cert" not found
Sep 07 09:18:31.495873 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.495853    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:31.497867 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.497846    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:31.514202 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.514178    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mvkw\" (UniqueName: \"kubernetes.io/projected/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-kube-api-access-4mvkw\") pod \"maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:31.570539 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.570457    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:31.898514 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.898438    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:31.998499 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.998460    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:31.999504 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:31.999479    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:32.001166 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:32.001148    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:32.205501 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:32.205461    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:32.479199 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:32.479130    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"]
Sep 07 09:18:32.482049 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:18:32.482011    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea5f9dc1_8fa0_4de1_a15c_5ca5bd848baa.slice/crio-1ec595c607f4d91f6c262b968c117118c063c8ed9eac4897b829efb00a2b91b6 WatchSource:0}: Error finding container 1ec595c607f4d91f6c262b968c117118c063c8ed9eac4897b829efb00a2b91b6: Status 404 returned error can't find the container with id 1ec595c607f4d91f6c262b968c117118c063c8ed9eac4897b829efb00a2b91b6
Sep 07 09:18:32.483427 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:32.483412    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 09:18:33.413790 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:33.413746    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk" event={"ID":"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa","Type":"ContainerStarted","Data":"0d89af4bfef86981c146aee668b2f4d9fc644b1b1f3161bfdc228988096183df"}
Sep 07 09:18:33.413993 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:33.413831    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk" event={"ID":"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa","Type":"ContainerStarted","Data":"1ec595c607f4d91f6c262b968c117118c063c8ed9eac4897b829efb00a2b91b6"}
Sep 07 09:18:33.413993 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:33.413872    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:33.429181 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:33.429128    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk" podStartSLOduration=2.042509898 podStartE2EDuration="2.429113537s" totalImagesPullingTime="386.603639ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:18:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:18:33.427590197 +0000 UTC m=+1196.391506434" watchObservedRunningTime="2026-09-07 09:18:33.429113537 +0000 UTC m=+1196.393029757"
Sep 07 09:18:34.371154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:34.371124    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:37.049625 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:37.049590    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:37.403094 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:37.402976    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:37.545550 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:37.545525    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:18:37.547345 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:37.547324    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:18:37.553943 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:37.553920    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:18:37.555408 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:37.555392    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:18:37.682939 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:37.681276    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:37.858344 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:37.858308    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:39.424277 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:39.424245    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:18:40.300975 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:40.300940    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:18:41.326170 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:41.326141    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:41.418164 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:41.418130    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:42.431320 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.431287    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"]
Sep 07 09:18:42.435014 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.434991    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:42.436806 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.436782    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-5560ef-serving-cert\""
Sep 07 09:18:42.444285 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.444261    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"]
Sep 07 09:18:42.464812 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.464779    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:42.482722 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.482688    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:42.482876 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.482771    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5dcn\" (UniqueName: \"kubernetes.io/projected/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-kube-api-access-n5dcn\") pod \"maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:42.482876 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.482860    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:42.482990 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.482895    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:42.564640 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.564614    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:42.583650 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.583622    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:42.583786 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.583672    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5dcn\" (UniqueName: \"kubernetes.io/projected/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-kube-api-access-n5dcn\") pod \"maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:42.583786 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.583742    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:42.583786 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.583765    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:42.583937 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:18:42.583869    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-serving-cert: secret "maas-api-e2e-shared-b-w2-5560ef-serving-cert" not found
Sep 07 09:18:42.583937 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:18:42.583934    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-maas-api-tls podName:9c8d56a7-4b52-40ce-80f9-03ec699f1b3a nodeName:}" failed. No retries permitted until 2026-09-07 09:18:43.083916974 +0000 UTC m=+1206.047833176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-maas-api-tls") pod "maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv" (UID: "9c8d56a7-4b52-40ce-80f9-03ec699f1b3a") : secret "maas-api-e2e-shared-b-w2-5560ef-serving-cert" not found
Sep 07 09:18:42.584470 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.584447    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:42.586398 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.586379    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:42.594107 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.594085    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5dcn\" (UniqueName: \"kubernetes.io/projected/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-kube-api-access-n5dcn\") pod \"maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:42.939601 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:42.939572    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:43.036604 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:43.036570    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:43.088297 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:43.088270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:43.090703 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:43.090680    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:43.110381 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:43.110353    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:43.347264 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:43.347188    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:43.617987 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:43.617964    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"]
Sep 07 09:18:43.619336 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:18:43.619313    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c8d56a7_4b52_40ce_80f9_03ec699f1b3a.slice/crio-e57328a9ddd4ec5e667c5a877bac7dd720e9acea4cdfc63fcd4733625e7fb1d2 WatchSource:0}: Error finding container e57328a9ddd4ec5e667c5a877bac7dd720e9acea4cdfc63fcd4733625e7fb1d2: Status 404 returned error can't find the container with id e57328a9ddd4ec5e667c5a877bac7dd720e9acea4cdfc63fcd4733625e7fb1d2
Sep 07 09:18:44.473541 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:44.473505    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv" event={"ID":"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a","Type":"ContainerStarted","Data":"f61c38425a1158bfc7dfb52d8cf23da46cc551d160d748c2081fe8a2b2929014"}
Sep 07 09:18:44.473760 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:44.473563    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:44.473760 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:44.473583    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv" event={"ID":"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a","Type":"ContainerStarted","Data":"e57328a9ddd4ec5e667c5a877bac7dd720e9acea4cdfc63fcd4733625e7fb1d2"}
Sep 07 09:18:44.491155 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:44.491110    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv" podStartSLOduration=2.103562747 podStartE2EDuration="2.49109718s" totalImagesPullingTime="387.534433ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:18:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:18:44.490020166 +0000 UTC m=+1207.453936435" watchObservedRunningTime="2026-09-07 09:18:44.49109718 +0000 UTC m=+1207.455013400"
Sep 07 09:18:50.485346 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:50.485315    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:18:52.000388 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:52.000353    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:18:53.417905 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:53.417866    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:55.193449 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:55.193413    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:55.524511 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:55.524427    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:55.640422 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:55.640391    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:55.838131 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:55.838042    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:56.693864 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:56.693827    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:59.347482 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:59.347446    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:18:59.442110 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:18:59.442078    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:00.669739 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.669694    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"]
Sep 07 09:19:00.673083 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.673067    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:00.678293 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.678268    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-34de95-serving-cert\""
Sep 07 09:19:00.693368 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.693345    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"]
Sep 07 09:19:00.706643 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.706612    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:00.745022 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.744989    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d06c2d4-8ee5-4325-a927-f370aa73858f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:00.745142 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.745036    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77fdg\" (UniqueName: \"kubernetes.io/projected/7d06c2d4-8ee5-4325-a927-f370aa73858f-kube-api-access-77fdg\") pod \"maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:00.745142 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.745109    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d06c2d4-8ee5-4325-a927-f370aa73858f-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:00.745233 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.745190    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d06c2d4-8ee5-4325-a927-f370aa73858f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:00.783788 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.783757    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:00.845827 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.845795    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d06c2d4-8ee5-4325-a927-f370aa73858f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:00.845952 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.845870    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d06c2d4-8ee5-4325-a927-f370aa73858f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:00.845952 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.845890    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77fdg\" (UniqueName: \"kubernetes.io/projected/7d06c2d4-8ee5-4325-a927-f370aa73858f-kube-api-access-77fdg\") pod \"maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:00.845952 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.845924    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d06c2d4-8ee5-4325-a927-f370aa73858f-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:00.846511 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.846490    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d06c2d4-8ee5-4325-a927-f370aa73858f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:00.848404 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.848385    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d06c2d4-8ee5-4325-a927-f370aa73858f-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:00.848823 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.848808    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d06c2d4-8ee5-4325-a927-f370aa73858f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:00.853022 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.853000    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77fdg\" (UniqueName: \"kubernetes.io/projected/7d06c2d4-8ee5-4325-a927-f370aa73858f-kube-api-access-77fdg\") pod \"maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:00.983819 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:00.983751    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:01.156200 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:01.156173    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:01.248389 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:01.248312    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:01.262842 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:01.262812    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"]
Sep 07 09:19:01.266263 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:19:01.266235    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d06c2d4_8ee5_4325_a927_f370aa73858f.slice/crio-fd4f50d608446ebbebea87877c33014c0e8a27f5c1bf8cdc6e0e5b7e6ebaa510 WatchSource:0}: Error finding container fd4f50d608446ebbebea87877c33014c0e8a27f5c1bf8cdc6e0e5b7e6ebaa510: Status 404 returned error can't find the container with id fd4f50d608446ebbebea87877c33014c0e8a27f5c1bf8cdc6e0e5b7e6ebaa510
Sep 07 09:19:01.564529 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:01.564447    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj" event={"ID":"7d06c2d4-8ee5-4325-a927-f370aa73858f","Type":"ContainerStarted","Data":"fd4f50d608446ebbebea87877c33014c0e8a27f5c1bf8cdc6e0e5b7e6ebaa510"}
Sep 07 09:19:02.571481 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:02.571448    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj" event={"ID":"7d06c2d4-8ee5-4325-a927-f370aa73858f","Type":"ContainerStarted","Data":"7bc31f9cc70f35dfdc4eaf41c979f72ca911dadcd5193c6f83fe0b9c20c1702d"}
Sep 07 09:19:02.571887 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:02.571529    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:02.587174 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:02.587138    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj" podStartSLOduration=2.224408463 podStartE2EDuration="2.587126985s" totalImagesPullingTime="362.718522ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:19:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:19:02.586238552 +0000 UTC m=+1225.550154791" watchObservedRunningTime="2026-09-07 09:19:02.587126985 +0000 UTC m=+1225.551043206"
Sep 07 09:19:03.321532 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:03.321501    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:05.302750 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:05.302697    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:19:06.297370 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:06.297333    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:08.581729 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:08.581684    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:19:19.467182 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:19.467150    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:19.823108 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:19.823027    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:20.032342 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:20.032305    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:22.419969 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:22.419924    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:22.573684 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:22.573651    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:22.667514 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:22.667478    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:23.148140 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:23.148104    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:23.433488 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:23.433452    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:23.530851 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:23.530827    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:23.824415 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:23.824317    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:23.869101 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:23.869073    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"]
Sep 07 09:19:23.873753 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:23.873737    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:23.875536 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:23.875517    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-12fc0fc8-serving-cert\""
Sep 07 09:19:23.883018 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:23.882995    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"]
Sep 07 09:19:23.894883 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:23.894846    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:23.954915 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:23.954891    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/395300ac-0387-4779-b64a-b37dfef318b6-metrics-certs\") pod \"maas-api-e2e-mt-12fc0fc8-674679986-q9nb6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:23.955033 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:23.954953    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/395300ac-0387-4779-b64a-b37dfef318b6-maas-api-tls\") pod \"maas-api-e2e-mt-12fc0fc8-674679986-q9nb6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:23.955033 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:23.955014    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz4k5\" (UniqueName: \"kubernetes.io/projected/395300ac-0387-4779-b64a-b37dfef318b6-kube-api-access-jz4k5\") pod \"maas-api-e2e-mt-12fc0fc8-674679986-q9nb6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:23.955111 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:23.955036    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/395300ac-0387-4779-b64a-b37dfef318b6-openshift-service-ca\") pod \"maas-api-e2e-mt-12fc0fc8-674679986-q9nb6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:23.959192 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:23.959168    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:24.055967 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.055942    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/395300ac-0387-4779-b64a-b37dfef318b6-maas-api-tls\") pod \"maas-api-e2e-mt-12fc0fc8-674679986-q9nb6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:24.056056 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.055978    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4k5\" (UniqueName: \"kubernetes.io/projected/395300ac-0387-4779-b64a-b37dfef318b6-kube-api-access-jz4k5\") pod \"maas-api-e2e-mt-12fc0fc8-674679986-q9nb6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:24.056056 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.055999    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/395300ac-0387-4779-b64a-b37dfef318b6-openshift-service-ca\") pod \"maas-api-e2e-mt-12fc0fc8-674679986-q9nb6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:24.056135 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.056072    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/395300ac-0387-4779-b64a-b37dfef318b6-metrics-certs\") pod \"maas-api-e2e-mt-12fc0fc8-674679986-q9nb6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:24.056705 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.056681    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/395300ac-0387-4779-b64a-b37dfef318b6-openshift-service-ca\") pod \"maas-api-e2e-mt-12fc0fc8-674679986-q9nb6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:24.058680 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.058658    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/395300ac-0387-4779-b64a-b37dfef318b6-maas-api-tls\") pod \"maas-api-e2e-mt-12fc0fc8-674679986-q9nb6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:24.058784 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.058703    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/395300ac-0387-4779-b64a-b37dfef318b6-metrics-certs\") pod \"maas-api-e2e-mt-12fc0fc8-674679986-q9nb6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:24.062993 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.062974    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz4k5\" (UniqueName: \"kubernetes.io/projected/395300ac-0387-4779-b64a-b37dfef318b6-kube-api-access-jz4k5\") pod \"maas-api-e2e-mt-12fc0fc8-674679986-q9nb6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:24.185767 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.185742    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:24.240018 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.239740    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:24.336375 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.336331    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:24.408265 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.408238    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:24.456450 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.456370    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"]
Sep 07 09:19:24.459233 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:19:24.459205    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395300ac_0387_4779_b64a_b37dfef318b6.slice/crio-151f5cb947fbcf50447c17e1d9247c720040b6fc4ae9715dbf3c1d0cd6a9cd4e WatchSource:0}: Error finding container 151f5cb947fbcf50447c17e1d9247c720040b6fc4ae9715dbf3c1d0cd6a9cd4e: Status 404 returned error can't find the container with id 151f5cb947fbcf50447c17e1d9247c720040b6fc4ae9715dbf3c1d0cd6a9cd4e
Sep 07 09:19:24.677143 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.677105    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6" event={"ID":"395300ac-0387-4779-b64a-b37dfef318b6","Type":"ContainerStarted","Data":"151f5cb947fbcf50447c17e1d9247c720040b6fc4ae9715dbf3c1d0cd6a9cd4e"}
Sep 07 09:19:24.746816 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:24.746739    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:25.501094 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:25.501056    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:25.683631 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:25.683600    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6" event={"ID":"395300ac-0387-4779-b64a-b37dfef318b6","Type":"ContainerStarted","Data":"93ac5102f772e043a92a8d57061c673b8a2a166cbac2104596f8c457aaed0c1a"}
Sep 07 09:19:25.683814 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:25.683740    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:25.704413 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:25.704366    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6" podStartSLOduration=2.288808787 podStartE2EDuration="2.704350292s" totalImagesPullingTime="415.541505ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:19:25.702360169 +0000 UTC m=+1248.666276407" watchObservedRunningTime="2026-09-07 09:19:25.704350292 +0000 UTC m=+1248.668266512"
Sep 07 09:19:28.501938 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:28.501861    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:19:31.693388 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:31.693358    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:19:31.900203 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:31.900165    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"]
Sep 07 09:19:31.904441 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:31.904417    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:31.906774 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:31.906749    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-3dc031\"/\"openshift-service-ca.crt\""
Sep 07 09:19:31.906915 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:31.906800    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-3dc031\"/\"kube-root-ca.crt\""
Sep 07 09:19:31.907007 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:31.906988    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-3dc031\"/\"auth-test-model-3dc031-kserve-self-signed-certs\""
Sep 07 09:19:31.907131 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:31.907028    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-3dc031\"/\"default-dockercfg-fm4xr\""
Sep 07 09:19:31.909907 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:31.909883    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"]
Sep 07 09:19:31.926064 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:31.926043    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:32.023914 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.023837    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-model-cache\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.023914 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.023871    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmzmx\" (UniqueName: \"kubernetes.io/projected/570c254f-8d8e-425a-aff2-5d64a83f5dc4-kube-api-access-zmzmx\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.023914 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.023895    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-tmp-dir\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.024117 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.023984    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-home\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.024117 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.024013    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-dshm\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.024117 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.024067    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/570c254f-8d8e-425a-aff2-5d64a83f5dc4-tls-certs\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.125216 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.125176    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-home\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.125216 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.125217    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-dshm\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.125435 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.125266    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/570c254f-8d8e-425a-aff2-5d64a83f5dc4-tls-certs\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.125435 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.125395    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-model-cache\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.125435 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.125430    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmzmx\" (UniqueName: \"kubernetes.io/projected/570c254f-8d8e-425a-aff2-5d64a83f5dc4-kube-api-access-zmzmx\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.125614 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.125474    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-tmp-dir\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.125614 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.125585    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-home\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.125827 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.125806    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-model-cache\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.125944 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.125832    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-tmp-dir\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.127777 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.127757    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-dshm\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.127952 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.127931    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/570c254f-8d8e-425a-aff2-5d64a83f5dc4-tls-certs\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.133551 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.133529    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmzmx\" (UniqueName: \"kubernetes.io/projected/570c254f-8d8e-425a-aff2-5d64a83f5dc4-kube-api-access-zmzmx\") pod \"auth-test-model-3dc031-kserve-548994c58c-xlkfv\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.216419 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.216393    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:32.485759 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.485736    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"]
Sep 07 09:19:32.487387 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:19:32.487364    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod570c254f_8d8e_425a_aff2_5d64a83f5dc4.slice/crio-cbac3ea3f8dc258d4739c7bc7829633b49e22e82573c0d5bb554dde068a6c45c WatchSource:0}: Error finding container cbac3ea3f8dc258d4739c7bc7829633b49e22e82573c0d5bb554dde068a6c45c: Status 404 returned error can't find the container with id cbac3ea3f8dc258d4739c7bc7829633b49e22e82573c0d5bb554dde068a6c45c
Sep 07 09:19:32.720534 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.720502    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv" event={"ID":"570c254f-8d8e-425a-aff2-5d64a83f5dc4","Type":"ContainerStarted","Data":"74ddccba271db5e5b1bb1009ba55b98ffd3ecd08c4a297d61bf84ee3f7f5dc01"}
Sep 07 09:19:32.720534 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.720540    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv" event={"ID":"570c254f-8d8e-425a-aff2-5d64a83f5dc4","Type":"ContainerStarted","Data":"cbac3ea3f8dc258d4739c7bc7829633b49e22e82573c0d5bb554dde068a6c45c"}
Sep 07 09:19:32.736175 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:32.736074    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv" podStartSLOduration=1.736055725 podStartE2EDuration="1.736055725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:19:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:19:32.735361638 +0000 UTC m=+1255.699277881" watchObservedRunningTime="2026-09-07 09:19:32.736055725 +0000 UTC m=+1255.699971947"
Sep 07 09:19:42.092814 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:42.092770    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:19:42.217482 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:42.217436    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:42.217633 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:42.217501    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:42.229807 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:42.229773    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:42.782476 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:42.782449    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:19:47.297858 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.297824    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"]
Sep 07 09:19:47.302815 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.302789    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.306134 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.305698    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-34de95\"/\"kube-root-ca.crt\""
Sep 07 09:19:47.306307 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.305957    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-34de95\"/\"auth-test-model-34de95-kserve-self-signed-certs\""
Sep 07 09:19:47.306424 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.306019    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-34de95\"/\"default-dockercfg-mflfq\""
Sep 07 09:19:47.306500 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.306033    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-34de95\"/\"openshift-service-ca.crt\""
Sep 07 09:19:47.309533 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.309213    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"]
Sep 07 09:19:47.331736 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.331696    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:47.454579 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.454536    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-tmp-dir\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.454769 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.454631    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-dshm\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.454769 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.454653    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksw97\" (UniqueName: \"kubernetes.io/projected/00ad294c-efd6-4332-a22b-4d3076eb13aa-kube-api-access-ksw97\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.454769 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.454673    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-model-cache\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.454769 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.454736    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ad294c-efd6-4332-a22b-4d3076eb13aa-tls-certs\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.454769 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.454766    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-home\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.555319 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.555249    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-model-cache\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.555319 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.555289    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ad294c-efd6-4332-a22b-4d3076eb13aa-tls-certs\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.555319 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.555312    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-home\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.555653 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.555350    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-tmp-dir\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.555653 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.555400    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-dshm\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.555653 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.555423    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksw97\" (UniqueName: \"kubernetes.io/projected/00ad294c-efd6-4332-a22b-4d3076eb13aa-kube-api-access-ksw97\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.555899 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.555655    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-model-cache\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.555899 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.555702    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-home\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.555899 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.555781    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-tmp-dir\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.557845 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.557825    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-dshm\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.557956 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.557939    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ad294c-efd6-4332-a22b-4d3076eb13aa-tls-certs\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.562765 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.562743    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksw97\" (UniqueName: \"kubernetes.io/projected/00ad294c-efd6-4332-a22b-4d3076eb13aa-kube-api-access-ksw97\") pod \"auth-test-model-34de95-kserve-86c489cd7-tqfr2\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") " pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.563816 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.563794    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:47.613625 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.613599    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:47.883294 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:47.883267    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"]
Sep 07 09:19:47.885351 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:19:47.885314    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00ad294c_efd6_4332_a22b_4d3076eb13aa.slice/crio-366cbc968cd5784a64405c61fee247004146479a64357b099093bef662c8744b WatchSource:0}: Error finding container 366cbc968cd5784a64405c61fee247004146479a64357b099093bef662c8744b: Status 404 returned error can't find the container with id 366cbc968cd5784a64405c61fee247004146479a64357b099093bef662c8744b
Sep 07 09:19:48.808381 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:48.808349    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2" event={"ID":"00ad294c-efd6-4332-a22b-4d3076eb13aa","Type":"ContainerStarted","Data":"de1157479bf6f28d16955d8abaf6518a9a36e555ea1bbb5ab1b9fcce1742f9e5"}
Sep 07 09:19:48.808776 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:48.808398    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2" event={"ID":"00ad294c-efd6-4332-a22b-4d3076eb13aa","Type":"ContainerStarted","Data":"366cbc968cd5784a64405c61fee247004146479a64357b099093bef662c8744b"}
Sep 07 09:19:48.824726 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:48.824669    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2" podStartSLOduration=1.8246526410000001 podStartE2EDuration="1.824652641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:19:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:19:48.824436386 +0000 UTC m=+1271.788352606" watchObservedRunningTime="2026-09-07 09:19:48.824652641 +0000 UTC m=+1271.788568861"
Sep 07 09:19:53.606966 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:53.606935    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:19:57.614277 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:57.614242    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:57.614277 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:57.614286    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:57.626588 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:57.626566    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:19:57.819947 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:57.819915    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:19:57.865020 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:19:57.864943    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:20:06.001642 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:20:06.001610    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:20:17.615731 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:20:17.615683    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:20:17.716169 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:20:17.716141    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:20:26.301509 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:20:26.301473    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:20:34.150753 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:20:34.150698    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:20:43.507024 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:20:43.506989    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:21:01.899905 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:01.899825    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:21:03.074972 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.074936    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"]
Sep 07 09:21:03.075451 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.075197    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6" podUID="395300ac-0387-4779-b64a-b37dfef318b6" containerName="maas-api" containerID="cri-o://93ac5102f772e043a92a8d57061c673b8a2a166cbac2104596f8c457aaed0c1a" gracePeriod=30
Sep 07 09:21:03.199770 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.199737    2574 generic.go:356] "Generic (PLEG): container finished" podID="395300ac-0387-4779-b64a-b37dfef318b6" containerID="93ac5102f772e043a92a8d57061c673b8a2a166cbac2104596f8c457aaed0c1a" exitCode=0
Sep 07 09:21:03.199882 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.199742    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6" event={"ID":"395300ac-0387-4779-b64a-b37dfef318b6","Type":"ContainerDied","Data":"93ac5102f772e043a92a8d57061c673b8a2a166cbac2104596f8c457aaed0c1a"}
Sep 07 09:21:03.219477 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.219372    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:03.292700 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.292661    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:03.331804 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.331748    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:21:03.356003 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.355088    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/395300ac-0387-4779-b64a-b37dfef318b6-openshift-service-ca\") pod \"395300ac-0387-4779-b64a-b37dfef318b6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") "
Sep 07 09:21:03.356003 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.355203    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/395300ac-0387-4779-b64a-b37dfef318b6-maas-api-tls\") pod \"395300ac-0387-4779-b64a-b37dfef318b6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") "
Sep 07 09:21:03.356003 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.355239    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jz4k5\" (UniqueName: \"kubernetes.io/projected/395300ac-0387-4779-b64a-b37dfef318b6-kube-api-access-jz4k5\") pod \"395300ac-0387-4779-b64a-b37dfef318b6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") "
Sep 07 09:21:03.356003 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.355273    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/395300ac-0387-4779-b64a-b37dfef318b6-metrics-certs\") pod \"395300ac-0387-4779-b64a-b37dfef318b6\" (UID: \"395300ac-0387-4779-b64a-b37dfef318b6\") "
Sep 07 09:21:03.357627 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.357510    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/395300ac-0387-4779-b64a-b37dfef318b6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "395300ac-0387-4779-b64a-b37dfef318b6" (UID: "395300ac-0387-4779-b64a-b37dfef318b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:21:03.361340 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.360040    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/395300ac-0387-4779-b64a-b37dfef318b6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "395300ac-0387-4779-b64a-b37dfef318b6" (UID: "395300ac-0387-4779-b64a-b37dfef318b6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:21:03.361340 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.360046    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/395300ac-0387-4779-b64a-b37dfef318b6-kube-api-access-jz4k5" (OuterVolumeSpecName: "kube-api-access-jz4k5") pod "395300ac-0387-4779-b64a-b37dfef318b6" (UID: "395300ac-0387-4779-b64a-b37dfef318b6"). InnerVolumeSpecName "kube-api-access-jz4k5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:21:03.362200 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.362151    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/395300ac-0387-4779-b64a-b37dfef318b6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "395300ac-0387-4779-b64a-b37dfef318b6" (UID: "395300ac-0387-4779-b64a-b37dfef318b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:21:03.456692 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.456648    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/395300ac-0387-4779-b64a-b37dfef318b6-openshift-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:21:03.456692 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.456690    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/395300ac-0387-4779-b64a-b37dfef318b6-maas-api-tls\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:21:03.456923 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.456737    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz4k5\" (UniqueName: \"kubernetes.io/projected/395300ac-0387-4779-b64a-b37dfef318b6-kube-api-access-jz4k5\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:21:03.456923 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:03.456752    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/395300ac-0387-4779-b64a-b37dfef318b6-metrics-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:21:04.207454 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:04.207426    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"
Sep 07 09:21:04.207911 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:04.207456    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6" event={"ID":"395300ac-0387-4779-b64a-b37dfef318b6","Type":"ContainerDied","Data":"151f5cb947fbcf50447c17e1d9247c720040b6fc4ae9715dbf3c1d0cd6a9cd4e"}
Sep 07 09:21:04.207911 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:04.207521    2574 scope.go:117] "RemoveContainer" containerID="93ac5102f772e043a92a8d57061c673b8a2a166cbac2104596f8c457aaed0c1a"
Sep 07 09:21:04.224911 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:04.224881    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"]
Sep 07 09:21:04.228149 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:04.228126    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-12fc0fc8-674679986-q9nb6"]
Sep 07 09:21:05.543000 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:05.542971    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="395300ac-0387-4779-b64a-b37dfef318b6" path="/var/lib/kubelet/pods/395300ac-0387-4779-b64a-b37dfef318b6/volumes"
Sep 07 09:21:06.182247 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:06.182208    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:11.204627 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:11.204591    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:20.199693 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:20.199660    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:21:35.991693 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:35.991660    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:21:47.717024 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:47.716975    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:48.186850 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:48.186816    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:49.273034 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:49.272999    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:51.493810 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:51.493778    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:21:53.590761 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:53.590731    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:53.656203 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:53.656168    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:53.843083 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:53.843004    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:55.119643 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:55.119605    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:55.218850 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:55.218808    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:55.319659 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:55.319622    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:55.916084 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:55.916037    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:56.017620 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:56.017583    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:56.119004 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:56.118962    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:21:57.322524 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:21:57.322486    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:03.181596 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:03.181560    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:03.244485 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:03.244451    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:03.837385 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:03.837348    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:07.904238 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:07.904195    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"]
Sep 07 09:22:07.918880 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:07.918846    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:08.907323 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:08.907290    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"]
Sep 07 09:22:08.932165 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:08.932127    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:09.768041 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:09.768001    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:09.937539 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:09.937494    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:10.029053 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:10.028973    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:10.502483 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:10.502451    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:22:20.437529 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:20.437494    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:20.670096 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:20.670053    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:22.927073 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:22.927029    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv" podUID="570c254f-8d8e-425a-aff2-5d64a83f5dc4" containerName="main" containerID="cri-o://74ddccba271db5e5b1bb1009ba55b98ffd3ecd08c4a297d61bf84ee3f7f5dc01" gracePeriod=45
Sep 07 09:22:23.186399 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.186341    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:22:23.275772 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.275744    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-model-cache\") pod \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") "
Sep 07 09:22:23.275914 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.275799    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-tmp-dir\") pod \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") "
Sep 07 09:22:23.275914 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.275824    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-home\") pod \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") "
Sep 07 09:22:23.275914 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.275857    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-dshm\") pod \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") "
Sep 07 09:22:23.276079 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.275923    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmzmx\" (UniqueName: \"kubernetes.io/projected/570c254f-8d8e-425a-aff2-5d64a83f5dc4-kube-api-access-zmzmx\") pod \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") "
Sep 07 09:22:23.276079 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.275989    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/570c254f-8d8e-425a-aff2-5d64a83f5dc4-tls-certs\") pod \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\" (UID: \"570c254f-8d8e-425a-aff2-5d64a83f5dc4\") "
Sep 07 09:22:23.276079 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.276056    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-model-cache" (OuterVolumeSpecName: "model-cache") pod "570c254f-8d8e-425a-aff2-5d64a83f5dc4" (UID: "570c254f-8d8e-425a-aff2-5d64a83f5dc4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:22:23.276079 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.276030    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "570c254f-8d8e-425a-aff2-5d64a83f5dc4" (UID: "570c254f-8d8e-425a-aff2-5d64a83f5dc4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:22:23.276279 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.276105    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-home" (OuterVolumeSpecName: "home") pod "570c254f-8d8e-425a-aff2-5d64a83f5dc4" (UID: "570c254f-8d8e-425a-aff2-5d64a83f5dc4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:22:23.276279 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.276267    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:22:23.276359 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.276290    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:22:23.276359 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.276305    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:22:23.278200 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.278169    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-dshm" (OuterVolumeSpecName: "dshm") pod "570c254f-8d8e-425a-aff2-5d64a83f5dc4" (UID: "570c254f-8d8e-425a-aff2-5d64a83f5dc4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:22:23.278310 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.278254    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/570c254f-8d8e-425a-aff2-5d64a83f5dc4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "570c254f-8d8e-425a-aff2-5d64a83f5dc4" (UID: "570c254f-8d8e-425a-aff2-5d64a83f5dc4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:22:23.278310 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.278282    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/570c254f-8d8e-425a-aff2-5d64a83f5dc4-kube-api-access-zmzmx" (OuterVolumeSpecName: "kube-api-access-zmzmx") pod "570c254f-8d8e-425a-aff2-5d64a83f5dc4" (UID: "570c254f-8d8e-425a-aff2-5d64a83f5dc4"). InnerVolumeSpecName "kube-api-access-zmzmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:22:23.377004 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.376969    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/570c254f-8d8e-425a-aff2-5d64a83f5dc4-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:22:23.377004 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.376996    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmzmx\" (UniqueName: \"kubernetes.io/projected/570c254f-8d8e-425a-aff2-5d64a83f5dc4-kube-api-access-zmzmx\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:22:23.377004 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.377006    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/570c254f-8d8e-425a-aff2-5d64a83f5dc4-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:22:23.497061 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.496983    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:22:23.634063 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.634032    2574 generic.go:356] "Generic (PLEG): container finished" podID="570c254f-8d8e-425a-aff2-5d64a83f5dc4" containerID="74ddccba271db5e5b1bb1009ba55b98ffd3ecd08c4a297d61bf84ee3f7f5dc01" exitCode=0
Sep 07 09:22:23.634206 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.634070    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"
Sep 07 09:22:23.634206 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.634119    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv" event={"ID":"570c254f-8d8e-425a-aff2-5d64a83f5dc4","Type":"ContainerDied","Data":"74ddccba271db5e5b1bb1009ba55b98ffd3ecd08c4a297d61bf84ee3f7f5dc01"}
Sep 07 09:22:23.634206 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.634161    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv" event={"ID":"570c254f-8d8e-425a-aff2-5d64a83f5dc4","Type":"ContainerDied","Data":"cbac3ea3f8dc258d4739c7bc7829633b49e22e82573c0d5bb554dde068a6c45c"}
Sep 07 09:22:23.634206 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.634183    2574 scope.go:117] "RemoveContainer" containerID="74ddccba271db5e5b1bb1009ba55b98ffd3ecd08c4a297d61bf84ee3f7f5dc01"
Sep 07 09:22:23.642976 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.642964    2574 scope.go:117] "RemoveContainer" containerID="74ddccba271db5e5b1bb1009ba55b98ffd3ecd08c4a297d61bf84ee3f7f5dc01"
Sep 07 09:22:23.643271 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:22:23.643254    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74ddccba271db5e5b1bb1009ba55b98ffd3ecd08c4a297d61bf84ee3f7f5dc01\": container with ID starting with 74ddccba271db5e5b1bb1009ba55b98ffd3ecd08c4a297d61bf84ee3f7f5dc01 not found: ID does not exist" containerID="74ddccba271db5e5b1bb1009ba55b98ffd3ecd08c4a297d61bf84ee3f7f5dc01"
Sep 07 09:22:23.643367 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.643276    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74ddccba271db5e5b1bb1009ba55b98ffd3ecd08c4a297d61bf84ee3f7f5dc01"} err="failed to get container status \"74ddccba271db5e5b1bb1009ba55b98ffd3ecd08c4a297d61bf84ee3f7f5dc01\": rpc error: code = NotFound desc = could not find container \"74ddccba271db5e5b1bb1009ba55b98ffd3ecd08c4a297d61bf84ee3f7f5dc01\": container with ID starting with 74ddccba271db5e5b1bb1009ba55b98ffd3ecd08c4a297d61bf84ee3f7f5dc01 not found: ID does not exist"
Sep 07 09:22:23.648302 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.648270    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"]
Sep 07 09:22:23.650923 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.650906    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/auth-test-model-3dc031-kserve-548994c58c-xlkfv"]
Sep 07 09:22:23.928442 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:23.928407    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2" podUID="00ad294c-efd6-4332-a22b-4d3076eb13aa" containerName="main" containerID="cri-o://de1157479bf6f28d16955d8abaf6518a9a36e555ea1bbb5ab1b9fcce1742f9e5" gracePeriod=45
Sep 07 09:22:24.183700 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.183645    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:22:24.284142 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.284114    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-dshm\") pod \"00ad294c-efd6-4332-a22b-4d3076eb13aa\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") "
Sep 07 09:22:24.284292 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.284166    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksw97\" (UniqueName: \"kubernetes.io/projected/00ad294c-efd6-4332-a22b-4d3076eb13aa-kube-api-access-ksw97\") pod \"00ad294c-efd6-4332-a22b-4d3076eb13aa\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") "
Sep 07 09:22:24.284292 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.284216    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ad294c-efd6-4332-a22b-4d3076eb13aa-tls-certs\") pod \"00ad294c-efd6-4332-a22b-4d3076eb13aa\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") "
Sep 07 09:22:24.284292 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.284248    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-home\") pod \"00ad294c-efd6-4332-a22b-4d3076eb13aa\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") "
Sep 07 09:22:24.284292 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.284266    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-model-cache\") pod \"00ad294c-efd6-4332-a22b-4d3076eb13aa\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") "
Sep 07 09:22:24.284521 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.284398    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-tmp-dir\") pod \"00ad294c-efd6-4332-a22b-4d3076eb13aa\" (UID: \"00ad294c-efd6-4332-a22b-4d3076eb13aa\") "
Sep 07 09:22:24.284521 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.284495    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-home" (OuterVolumeSpecName: "home") pod "00ad294c-efd6-4332-a22b-4d3076eb13aa" (UID: "00ad294c-efd6-4332-a22b-4d3076eb13aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:22:24.284521 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.284508    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-model-cache" (OuterVolumeSpecName: "model-cache") pod "00ad294c-efd6-4332-a22b-4d3076eb13aa" (UID: "00ad294c-efd6-4332-a22b-4d3076eb13aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:22:24.284651 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.284629    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "00ad294c-efd6-4332-a22b-4d3076eb13aa" (UID: "00ad294c-efd6-4332-a22b-4d3076eb13aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:22:24.284790 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.284760    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:22:24.284790 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.284786    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:22:24.285150 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.284799    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:22:24.286522 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.286500    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-dshm" (OuterVolumeSpecName: "dshm") pod "00ad294c-efd6-4332-a22b-4d3076eb13aa" (UID: "00ad294c-efd6-4332-a22b-4d3076eb13aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:22:24.286609 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.286559    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00ad294c-efd6-4332-a22b-4d3076eb13aa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "00ad294c-efd6-4332-a22b-4d3076eb13aa" (UID: "00ad294c-efd6-4332-a22b-4d3076eb13aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:22:24.286652 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.286624    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00ad294c-efd6-4332-a22b-4d3076eb13aa-kube-api-access-ksw97" (OuterVolumeSpecName: "kube-api-access-ksw97") pod "00ad294c-efd6-4332-a22b-4d3076eb13aa" (UID: "00ad294c-efd6-4332-a22b-4d3076eb13aa"). InnerVolumeSpecName "kube-api-access-ksw97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:22:24.386006 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.385978    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00ad294c-efd6-4332-a22b-4d3076eb13aa-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:22:24.386006 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.386003    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksw97\" (UniqueName: \"kubernetes.io/projected/00ad294c-efd6-4332-a22b-4d3076eb13aa-kube-api-access-ksw97\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:22:24.386188 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.386015    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00ad294c-efd6-4332-a22b-4d3076eb13aa-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:22:24.642326 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.642234    2574 generic.go:356] "Generic (PLEG): container finished" podID="00ad294c-efd6-4332-a22b-4d3076eb13aa" containerID="de1157479bf6f28d16955d8abaf6518a9a36e555ea1bbb5ab1b9fcce1742f9e5" exitCode=0
Sep 07 09:22:24.642326 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.642293    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"
Sep 07 09:22:24.642326 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.642303    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2" event={"ID":"00ad294c-efd6-4332-a22b-4d3076eb13aa","Type":"ContainerDied","Data":"de1157479bf6f28d16955d8abaf6518a9a36e555ea1bbb5ab1b9fcce1742f9e5"}
Sep 07 09:22:24.642596 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.642340    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2" event={"ID":"00ad294c-efd6-4332-a22b-4d3076eb13aa","Type":"ContainerDied","Data":"366cbc968cd5784a64405c61fee247004146479a64357b099093bef662c8744b"}
Sep 07 09:22:24.642596 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.642362    2574 scope.go:117] "RemoveContainer" containerID="de1157479bf6f28d16955d8abaf6518a9a36e555ea1bbb5ab1b9fcce1742f9e5"
Sep 07 09:22:24.652066 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.652043    2574 scope.go:117] "RemoveContainer" containerID="de1157479bf6f28d16955d8abaf6518a9a36e555ea1bbb5ab1b9fcce1742f9e5"
Sep 07 09:22:24.652334 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:22:24.652315    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de1157479bf6f28d16955d8abaf6518a9a36e555ea1bbb5ab1b9fcce1742f9e5\": container with ID starting with de1157479bf6f28d16955d8abaf6518a9a36e555ea1bbb5ab1b9fcce1742f9e5 not found: ID does not exist" containerID="de1157479bf6f28d16955d8abaf6518a9a36e555ea1bbb5ab1b9fcce1742f9e5"
Sep 07 09:22:24.652400 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.652343    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de1157479bf6f28d16955d8abaf6518a9a36e555ea1bbb5ab1b9fcce1742f9e5"} err="failed to get container status \"de1157479bf6f28d16955d8abaf6518a9a36e555ea1bbb5ab1b9fcce1742f9e5\": rpc error: code = NotFound desc = could not find container \"de1157479bf6f28d16955d8abaf6518a9a36e555ea1bbb5ab1b9fcce1742f9e5\": container with ID starting with de1157479bf6f28d16955d8abaf6518a9a36e555ea1bbb5ab1b9fcce1742f9e5 not found: ID does not exist"
Sep 07 09:22:24.662881 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.662855    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"]
Sep 07 09:22:24.664770 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:24.664751    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/auth-test-model-34de95-kserve-86c489cd7-tqfr2"]
Sep 07 09:22:25.542443 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:25.542411    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00ad294c-efd6-4332-a22b-4d3076eb13aa" path="/var/lib/kubelet/pods/00ad294c-efd6-4332-a22b-4d3076eb13aa/volumes"
Sep 07 09:22:25.542996 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:25.542975    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="570c254f-8d8e-425a-aff2-5d64a83f5dc4" path="/var/lib/kubelet/pods/570c254f-8d8e-425a-aff2-5d64a83f5dc4/volumes"
Sep 07 09:22:28.102013 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.101978    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"]
Sep 07 09:22:28.102378 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.102364    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="570c254f-8d8e-425a-aff2-5d64a83f5dc4" containerName="main"
Sep 07 09:22:28.102378 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.102376    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="570c254f-8d8e-425a-aff2-5d64a83f5dc4" containerName="main"
Sep 07 09:22:28.102452 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.102395    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00ad294c-efd6-4332-a22b-4d3076eb13aa" containerName="main"
Sep 07 09:22:28.102452 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.102400    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="00ad294c-efd6-4332-a22b-4d3076eb13aa" containerName="main"
Sep 07 09:22:28.102452 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.102408    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="395300ac-0387-4779-b64a-b37dfef318b6" containerName="maas-api"
Sep 07 09:22:28.102452 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.102414    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="395300ac-0387-4779-b64a-b37dfef318b6" containerName="maas-api"
Sep 07 09:22:28.102579 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.102472    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="00ad294c-efd6-4332-a22b-4d3076eb13aa" containerName="main"
Sep 07 09:22:28.102579 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.102482    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="570c254f-8d8e-425a-aff2-5d64a83f5dc4" containerName="main"
Sep 07 09:22:28.102579 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.102488    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="395300ac-0387-4779-b64a-b37dfef318b6" containerName="maas-api"
Sep 07 09:22:28.106889 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.106869    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.108829 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.108811    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-3dc031\"/\"kube-root-ca.crt\""
Sep 07 09:22:28.109283 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.109259    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-3dc031\"/\"default-dockercfg-fm4xr\""
Sep 07 09:22:28.109378 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.109340    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-3dc031\"/\"sub-test-model-3dc031-kserve-self-signed-certs\""
Sep 07 09:22:28.109378 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.109353    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-3dc031\"/\"openshift-service-ca.crt\""
Sep 07 09:22:28.114931 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.114619    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-model-cache\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.114931 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.114660    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hprw\" (UniqueName: \"kubernetes.io/projected/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-kube-api-access-9hprw\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.114931 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.114688    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-tls-certs\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.114931 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.114740    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-tmp-dir\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.114931 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.114807    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-dshm\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.114931 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.114867    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-home\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.115550 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.115530    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"]
Sep 07 09:22:28.202304 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.202267    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:28.215494 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.215472    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-home\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.215591 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.215535    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-model-cache\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.215591 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.215567    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hprw\" (UniqueName: \"kubernetes.io/projected/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-kube-api-access-9hprw\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.215695 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.215594    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-tls-certs\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.215695 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.215623    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-tmp-dir\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.215695 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.215675    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-dshm\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.215959 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.215929    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-home\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.216031 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.215966    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-model-cache\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.216068 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.216024    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-tmp-dir\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.218019 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.218000    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-dshm\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.218288 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.218271    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-tls-certs\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.222742 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.222703    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hprw\" (UniqueName: \"kubernetes.io/projected/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-kube-api-access-9hprw\") pod \"sub-test-model-3dc031-kserve-67658b9899-xwrbz\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.418012 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.417971    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:28.646282 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.646253    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:28.689478 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.689386    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"]
Sep 07 09:22:28.692061 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:22:28.692031    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf15ee100_c58a_4ff6_85ae_3f52dd3ba762.slice/crio-36f03fffc2d84892d151411cd4dbc00f0918ae8aed485bba83365124e02d8a04 WatchSource:0}: Error finding container 36f03fffc2d84892d151411cd4dbc00f0918ae8aed485bba83365124e02d8a04: Status 404 returned error can't find the container with id 36f03fffc2d84892d151411cd4dbc00f0918ae8aed485bba83365124e02d8a04
Sep 07 09:22:28.748016 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:28.747980    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:29.673269 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:29.673233    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz" event={"ID":"f15ee100-c58a-4ff6-85ae-3f52dd3ba762","Type":"ContainerStarted","Data":"93a1d715dd316b5ff427c4367fac52d9b7afd08dfd35bd46b25936d539e0d190"}
Sep 07 09:22:29.673640 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:29.673278    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz" event={"ID":"f15ee100-c58a-4ff6-85ae-3f52dd3ba762","Type":"ContainerStarted","Data":"36f03fffc2d84892d151411cd4dbc00f0918ae8aed485bba83365124e02d8a04"}
Sep 07 09:22:29.689181 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:29.689140    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz" podStartSLOduration=1.689125263 podStartE2EDuration="1.689125263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:22:29.687845506 +0000 UTC m=+1432.651761748" watchObservedRunningTime="2026-09-07 09:22:29.689125263 +0000 UTC m=+1432.653041525"
Sep 07 09:22:32.700553 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:32.700520    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:22:38.418794 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:38.418762    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:38.419164 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:38.418806    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:38.431501 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:38.431475    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:38.729283 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:38.729208    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:22:41.588119 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:41.588088    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:47.533869 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:47.533827    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:47.595735 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:47.595675    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:47.855994 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:47.855902    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:48.161759 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:48.161726    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:48.804547 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:48.804514    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:49.399729 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.399682    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"]
Sep 07 09:22:49.404745 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.404726    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.407130 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.407106    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-34de95\"/\"default-dockercfg-mflfq\""
Sep 07 09:22:49.407130 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.407123    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-34de95\"/\"sub-test-model-34de95-kserve-self-signed-certs\""
Sep 07 09:22:49.407313 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.407123    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-34de95\"/\"kube-root-ca.crt\""
Sep 07 09:22:49.407313 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.407113    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-34de95\"/\"openshift-service-ca.crt\""
Sep 07 09:22:49.411060 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.411036    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"]
Sep 07 09:22:49.423012 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.422984    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:49.499381 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.499346    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-home\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.499381 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.499383    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-dshm\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.499585 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.499408    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgm9h\" (UniqueName: \"kubernetes.io/projected/4c604300-7c74-4568-915f-de5b7151a94c-kube-api-access-dgm9h\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.499585 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.499539    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c604300-7c74-4568-915f-de5b7151a94c-tls-certs\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.499675 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.499601    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-tmp-dir\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.499675 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.499640    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-model-cache\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.600263 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.600231    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-tmp-dir\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.600263 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.600266    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-model-cache\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.600507 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.600328    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-home\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.600507 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.600356    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-dshm\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.600507 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.600388    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgm9h\" (UniqueName: \"kubernetes.io/projected/4c604300-7c74-4568-915f-de5b7151a94c-kube-api-access-dgm9h\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.600507 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.600494    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c604300-7c74-4568-915f-de5b7151a94c-tls-certs\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.600763 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.600654    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-tmp-dir\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.600763 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.600742    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-home\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.600923 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.600902    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-model-cache\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.602843 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.602821    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-dshm\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.603103 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.603084    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c604300-7c74-4568-915f-de5b7151a94c-tls-certs\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.607326 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.607304    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgm9h\" (UniqueName: \"kubernetes.io/projected/4c604300-7c74-4568-915f-de5b7151a94c-kube-api-access-dgm9h\") pod \"sub-test-model-34de95-kserve-7456489b8c-f4vd6\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") " pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.715017 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.714987    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:49.985272 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:49.985245    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"]
Sep 07 09:22:49.987079 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:22:49.987053    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c604300_7c74_4568_915f_de5b7151a94c.slice/crio-6aa1c1232396523807f06ceb63724068d5a2a9508ad8a7ea0321863b33b159c8 WatchSource:0}: Error finding container 6aa1c1232396523807f06ceb63724068d5a2a9508ad8a7ea0321863b33b159c8: Status 404 returned error can't find the container with id 6aa1c1232396523807f06ceb63724068d5a2a9508ad8a7ea0321863b33b159c8
Sep 07 09:22:50.785425 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:50.785375    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6" event={"ID":"4c604300-7c74-4568-915f-de5b7151a94c","Type":"ContainerStarted","Data":"11f8b9a82498142322857bec0c0977b3b519b3365bc4734454d902f58d4dbd50"}
Sep 07 09:22:50.785640 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:50.785435    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6" event={"ID":"4c604300-7c74-4568-915f-de5b7151a94c","Type":"ContainerStarted","Data":"6aa1c1232396523807f06ceb63724068d5a2a9508ad8a7ea0321863b33b159c8"}
Sep 07 09:22:50.796998 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:50.796964    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:50.808541 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:50.808499    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6" podStartSLOduration=1.808485906 podStartE2EDuration="1.808485906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:22:50.806465432 +0000 UTC m=+1453.770381661" watchObservedRunningTime="2026-09-07 09:22:50.808485906 +0000 UTC m=+1453.772402172"
Sep 07 09:22:50.891400 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:50.891362    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:51.932695 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:51.932661    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:52.033633 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:52.033584    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:52.178126 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:52.178094    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:52.441358 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:52.441329    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:52.542546 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:52.542514    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:22:53.599067 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:53.599036    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:22:59.716027 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:59.715997    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:59.716497 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:59.716054    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:59.729824 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:59.729799    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:22:59.843858 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:22:59.843828    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:23:09.201656 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:09.201619    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:23:15.746380 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:15.746342    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:16.174975 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:16.174939    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:16.454026 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:16.453939    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:16.629567 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:16.629534    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:19.997261 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:19.997233    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:20.098011 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:20.097975    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:21.158736 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.158668    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"]
Sep 07 09:23:21.162925 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.162902    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:21.165315 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.165296    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-558343-serving-cert\""
Sep 07 09:23:21.183477 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.183452    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"]
Sep 07 09:23:21.193345 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.193320    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-maas-api-tls\") pod \"maas-api-e2e-api-b-558343-77d575bb59-md4b9\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:21.193478 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.193357    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/834e08cf-4e50-4cf3-958b-673f1ce53a34-openshift-service-ca\") pod \"maas-api-e2e-api-b-558343-77d575bb59-md4b9\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:21.193478 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.193462    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxrt4\" (UniqueName: \"kubernetes.io/projected/834e08cf-4e50-4cf3-958b-673f1ce53a34-kube-api-access-cxrt4\") pod \"maas-api-e2e-api-b-558343-77d575bb59-md4b9\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:21.193600 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.193509    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-metrics-certs\") pod \"maas-api-e2e-api-b-558343-77d575bb59-md4b9\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:21.211948 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.211912    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:21.294704 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.294680    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/834e08cf-4e50-4cf3-958b-673f1ce53a34-openshift-service-ca\") pod \"maas-api-e2e-api-b-558343-77d575bb59-md4b9\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:21.294844 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.294738    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxrt4\" (UniqueName: \"kubernetes.io/projected/834e08cf-4e50-4cf3-958b-673f1ce53a34-kube-api-access-cxrt4\") pod \"maas-api-e2e-api-b-558343-77d575bb59-md4b9\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:21.294912 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.294846    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-metrics-certs\") pod \"maas-api-e2e-api-b-558343-77d575bb59-md4b9\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:21.294983 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.294969    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-maas-api-tls\") pod \"maas-api-e2e-api-b-558343-77d575bb59-md4b9\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:21.295079 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:23:21.295065    2574 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-558343-serving-cert: secret "maas-api-e2e-api-b-558343-serving-cert" not found
Sep 07 09:23:21.295142 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:23:21.295118    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-maas-api-tls podName:834e08cf-4e50-4cf3-958b-673f1ce53a34 nodeName:}" failed. No retries permitted until 2026-09-07 09:23:21.795103401 +0000 UTC m=+1484.759019614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-maas-api-tls") pod "maas-api-e2e-api-b-558343-77d575bb59-md4b9" (UID: "834e08cf-4e50-4cf3-958b-673f1ce53a34") : secret "maas-api-e2e-api-b-558343-serving-cert" not found
Sep 07 09:23:21.295292 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.295274    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/834e08cf-4e50-4cf3-958b-673f1ce53a34-openshift-service-ca\") pod \"maas-api-e2e-api-b-558343-77d575bb59-md4b9\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:21.297314 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.297296    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-metrics-certs\") pod \"maas-api-e2e-api-b-558343-77d575bb59-md4b9\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:21.305802 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.305783    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxrt4\" (UniqueName: \"kubernetes.io/projected/834e08cf-4e50-4cf3-958b-673f1ce53a34-kube-api-access-cxrt4\") pod \"maas-api-e2e-api-b-558343-77d575bb59-md4b9\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:21.308465 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.308442    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:21.481362 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.481269    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:21.739257 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.739186    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:21.798997 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.798967    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-maas-api-tls\") pod \"maas-api-e2e-api-b-558343-77d575bb59-md4b9\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:21.801445 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.801416    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-maas-api-tls\") pod \"maas-api-e2e-api-b-558343-77d575bb59-md4b9\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:21.822381 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:21.822352    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:22.075087 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:22.075003    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:22.346936 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:22.346913    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"]
Sep 07 09:23:22.348847 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:23:22.348819    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod834e08cf_4e50_4cf3_958b_673f1ce53a34.slice/crio-ba569a4eded592c766f66b109efbc16fea124e823d078059591c95131301e712 WatchSource:0}: Error finding container ba569a4eded592c766f66b109efbc16fea124e823d078059591c95131301e712: Status 404 returned error can't find the container with id ba569a4eded592c766f66b109efbc16fea124e823d078059591c95131301e712
Sep 07 09:23:22.951195 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:22.951166    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9" event={"ID":"834e08cf-4e50-4cf3-958b-673f1ce53a34","Type":"ContainerStarted","Data":"f9c386523c304842996ece01852b6e6c71f35a4b6b05494c378f19693553d263"}
Sep 07 09:23:22.951377 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:22.951226    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9" event={"ID":"834e08cf-4e50-4cf3-958b-673f1ce53a34","Type":"ContainerStarted","Data":"ba569a4eded592c766f66b109efbc16fea124e823d078059591c95131301e712"}
Sep 07 09:23:22.951377 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:22.951258    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:22.968182 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:22.968138    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9" podStartSLOduration=1.4961038389999999 podStartE2EDuration="1.968121051s" totalImagesPullingTime="472.017212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:23:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:23:22.967277975 +0000 UTC m=+1485.931194195" watchObservedRunningTime="2026-09-07 09:23:22.968121051 +0000 UTC m=+1485.932037271"
Sep 07 09:23:26.700611 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:26.700580    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:23:28.962757 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:28.962700    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:23:35.702152 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:35.702122    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:37.576780 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:37.576746    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:23:37.579334 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:37.579315    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:23:37.584217 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:37.584198    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:23:37.586615 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:37.586601    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:23:37.677999 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:37.677971    2574 scope.go:117] "RemoveContainer" containerID="e624295bce96ae72a907975eca8291d39b428bc064f3d31b6e6290ae87cd984a"
Sep 07 09:23:42.102901 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:42.102858    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:44.392362 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:44.392328    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:23:44.925737 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:44.925679    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:47.812486 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:47.812451    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:48.107832 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:48.107760    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"]
Sep 07 09:23:48.131897 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:48.131869    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:49.090539 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:49.090506    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"]
Sep 07 09:23:49.112390 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:49.112360    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:54.102697 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:54.102661    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:23:57.910970 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:23:57.910939    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:01.600221 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:01.600188    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:24:03.127899 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.127858    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz" podUID="f15ee100-c58a-4ff6-85ae-3f52dd3ba762" containerName="main" containerID="cri-o://93a1d715dd316b5ff427c4367fac52d9b7afd08dfd35bd46b25936d539e0d190" gracePeriod=45
Sep 07 09:24:03.383788 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.383696    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:24:03.411328 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.411300    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:03.469330 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.469294    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hprw\" (UniqueName: \"kubernetes.io/projected/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-kube-api-access-9hprw\") pod \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") "
Sep 07 09:24:03.469330 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.469347    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-dshm\") pod \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") "
Sep 07 09:24:03.469612 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.469471    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-tls-certs\") pod \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") "
Sep 07 09:24:03.469612 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.469528    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-home\") pod \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") "
Sep 07 09:24:03.469612 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.469581    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-tmp-dir\") pod \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") "
Sep 07 09:24:03.469898 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.469613    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-model-cache\") pod \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\" (UID: \"f15ee100-c58a-4ff6-85ae-3f52dd3ba762\") "
Sep 07 09:24:03.469957 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.469927    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-home" (OuterVolumeSpecName: "home") pod "f15ee100-c58a-4ff6-85ae-3f52dd3ba762" (UID: "f15ee100-c58a-4ff6-85ae-3f52dd3ba762"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:24:03.470091 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.470049    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-model-cache" (OuterVolumeSpecName: "model-cache") pod "f15ee100-c58a-4ff6-85ae-3f52dd3ba762" (UID: "f15ee100-c58a-4ff6-85ae-3f52dd3ba762"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:24:03.470180 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.470128    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f15ee100-c58a-4ff6-85ae-3f52dd3ba762" (UID: "f15ee100-c58a-4ff6-85ae-3f52dd3ba762"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:24:03.472164 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.472140    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f15ee100-c58a-4ff6-85ae-3f52dd3ba762" (UID: "f15ee100-c58a-4ff6-85ae-3f52dd3ba762"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:24:03.472337 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.472311    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-kube-api-access-9hprw" (OuterVolumeSpecName: "kube-api-access-9hprw") pod "f15ee100-c58a-4ff6-85ae-3f52dd3ba762" (UID: "f15ee100-c58a-4ff6-85ae-3f52dd3ba762"). InnerVolumeSpecName "kube-api-access-9hprw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:24:03.472480 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.472389    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-dshm" (OuterVolumeSpecName: "dshm") pod "f15ee100-c58a-4ff6-85ae-3f52dd3ba762" (UID: "f15ee100-c58a-4ff6-85ae-3f52dd3ba762"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:24:03.477500 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.477473    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:03.549670 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.549641    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:03.570688 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.570666    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:03.570688 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.570685    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:03.570841 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.570694    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:03.570841 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.570703    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hprw\" (UniqueName: \"kubernetes.io/projected/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-kube-api-access-9hprw\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:03.570841 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.570728    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:03.570841 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.570736    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f15ee100-c58a-4ff6-85ae-3f52dd3ba762-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:03.611904 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.611877    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:03.912111 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:03.912070    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:04.011629 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.011602    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:04.110839 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.110805    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6" podUID="4c604300-7c74-4568-915f-de5b7151a94c" containerName="main" containerID="cri-o://11f8b9a82498142322857bec0c0977b3b519b3365bc4734454d902f58d4dbd50" gracePeriod=45
Sep 07 09:24:04.165848 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.165778    2574 generic.go:356] "Generic (PLEG): container finished" podID="f15ee100-c58a-4ff6-85ae-3f52dd3ba762" containerID="93a1d715dd316b5ff427c4367fac52d9b7afd08dfd35bd46b25936d539e0d190" exitCode=0
Sep 07 09:24:04.165848 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.165820    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"
Sep 07 09:24:04.166234 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.165865    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz" event={"ID":"f15ee100-c58a-4ff6-85ae-3f52dd3ba762","Type":"ContainerDied","Data":"93a1d715dd316b5ff427c4367fac52d9b7afd08dfd35bd46b25936d539e0d190"}
Sep 07 09:24:04.166234 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.165915    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz" event={"ID":"f15ee100-c58a-4ff6-85ae-3f52dd3ba762","Type":"ContainerDied","Data":"36f03fffc2d84892d151411cd4dbc00f0918ae8aed485bba83365124e02d8a04"}
Sep 07 09:24:04.166234 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.165933    2574 scope.go:117] "RemoveContainer" containerID="93a1d715dd316b5ff427c4367fac52d9b7afd08dfd35bd46b25936d539e0d190"
Sep 07 09:24:04.191429 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.191406    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"]
Sep 07 09:24:04.196829 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.196810    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/sub-test-model-3dc031-kserve-67658b9899-xwrbz"]
Sep 07 09:24:04.222057 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.222033    2574 scope.go:117] "RemoveContainer" containerID="93a1d715dd316b5ff427c4367fac52d9b7afd08dfd35bd46b25936d539e0d190"
Sep 07 09:24:04.222414 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:24:04.222390    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93a1d715dd316b5ff427c4367fac52d9b7afd08dfd35bd46b25936d539e0d190\": container with ID starting with 93a1d715dd316b5ff427c4367fac52d9b7afd08dfd35bd46b25936d539e0d190 not found: ID does not exist" containerID="93a1d715dd316b5ff427c4367fac52d9b7afd08dfd35bd46b25936d539e0d190"
Sep 07 09:24:04.222468 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.222423    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93a1d715dd316b5ff427c4367fac52d9b7afd08dfd35bd46b25936d539e0d190"} err="failed to get container status \"93a1d715dd316b5ff427c4367fac52d9b7afd08dfd35bd46b25936d539e0d190\": rpc error: code = NotFound desc = could not find container \"93a1d715dd316b5ff427c4367fac52d9b7afd08dfd35bd46b25936d539e0d190\": container with ID starting with 93a1d715dd316b5ff427c4367fac52d9b7afd08dfd35bd46b25936d539e0d190 not found: ID does not exist"
Sep 07 09:24:04.379383 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.379352    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:24:04.479942 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.479848    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c604300-7c74-4568-915f-de5b7151a94c-tls-certs\") pod \"4c604300-7c74-4568-915f-de5b7151a94c\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") "
Sep 07 09:24:04.479942 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.479888    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgm9h\" (UniqueName: \"kubernetes.io/projected/4c604300-7c74-4568-915f-de5b7151a94c-kube-api-access-dgm9h\") pod \"4c604300-7c74-4568-915f-de5b7151a94c\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") "
Sep 07 09:24:04.479942 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.479919    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-home\") pod \"4c604300-7c74-4568-915f-de5b7151a94c\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") "
Sep 07 09:24:04.479942 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.479938    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-dshm\") pod \"4c604300-7c74-4568-915f-de5b7151a94c\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") "
Sep 07 09:24:04.480284 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.479978    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-model-cache\") pod \"4c604300-7c74-4568-915f-de5b7151a94c\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") "
Sep 07 09:24:04.480284 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.480035    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-tmp-dir\") pod \"4c604300-7c74-4568-915f-de5b7151a94c\" (UID: \"4c604300-7c74-4568-915f-de5b7151a94c\") "
Sep 07 09:24:04.480284 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.480233    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-home" (OuterVolumeSpecName: "home") pod "4c604300-7c74-4568-915f-de5b7151a94c" (UID: "4c604300-7c74-4568-915f-de5b7151a94c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:24:04.480406 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.480281    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c604300-7c74-4568-915f-de5b7151a94c" (UID: "4c604300-7c74-4568-915f-de5b7151a94c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:24:04.480406 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.480297    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c604300-7c74-4568-915f-de5b7151a94c" (UID: "4c604300-7c74-4568-915f-de5b7151a94c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:24:04.482258 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.482237    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c604300-7c74-4568-915f-de5b7151a94c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c604300-7c74-4568-915f-de5b7151a94c" (UID: "4c604300-7c74-4568-915f-de5b7151a94c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:24:04.482258 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.482244    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-dshm" (OuterVolumeSpecName: "dshm") pod "4c604300-7c74-4568-915f-de5b7151a94c" (UID: "4c604300-7c74-4568-915f-de5b7151a94c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:24:04.482395 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.482294    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c604300-7c74-4568-915f-de5b7151a94c-kube-api-access-dgm9h" (OuterVolumeSpecName: "kube-api-access-dgm9h") pod "4c604300-7c74-4568-915f-de5b7151a94c" (UID: "4c604300-7c74-4568-915f-de5b7151a94c"). InnerVolumeSpecName "kube-api-access-dgm9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:24:04.580771 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.580732    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:04.580771 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.580764    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c604300-7c74-4568-915f-de5b7151a94c-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:04.580959 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.580778    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgm9h\" (UniqueName: \"kubernetes.io/projected/4c604300-7c74-4568-915f-de5b7151a94c-kube-api-access-dgm9h\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:04.580959 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.580789    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:04.580959 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.580799    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:04.580959 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:04.580812    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c604300-7c74-4568-915f-de5b7151a94c-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:05.172751 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:05.172695    2574 generic.go:356] "Generic (PLEG): container finished" podID="4c604300-7c74-4568-915f-de5b7151a94c" containerID="11f8b9a82498142322857bec0c0977b3b519b3365bc4734454d902f58d4dbd50" exitCode=0
Sep 07 09:24:05.173154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:05.172763    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"
Sep 07 09:24:05.173154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:05.172786    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6" event={"ID":"4c604300-7c74-4568-915f-de5b7151a94c","Type":"ContainerDied","Data":"11f8b9a82498142322857bec0c0977b3b519b3365bc4734454d902f58d4dbd50"}
Sep 07 09:24:05.173154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:05.172833    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6" event={"ID":"4c604300-7c74-4568-915f-de5b7151a94c","Type":"ContainerDied","Data":"6aa1c1232396523807f06ceb63724068d5a2a9508ad8a7ea0321863b33b159c8"}
Sep 07 09:24:05.173154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:05.172855    2574 scope.go:117] "RemoveContainer" containerID="11f8b9a82498142322857bec0c0977b3b519b3365bc4734454d902f58d4dbd50"
Sep 07 09:24:05.182207 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:05.182191    2574 scope.go:117] "RemoveContainer" containerID="11f8b9a82498142322857bec0c0977b3b519b3365bc4734454d902f58d4dbd50"
Sep 07 09:24:05.182430 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:24:05.182410    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11f8b9a82498142322857bec0c0977b3b519b3365bc4734454d902f58d4dbd50\": container with ID starting with 11f8b9a82498142322857bec0c0977b3b519b3365bc4734454d902f58d4dbd50 not found: ID does not exist" containerID="11f8b9a82498142322857bec0c0977b3b519b3365bc4734454d902f58d4dbd50"
Sep 07 09:24:05.182498 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:05.182441    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11f8b9a82498142322857bec0c0977b3b519b3365bc4734454d902f58d4dbd50"} err="failed to get container status \"11f8b9a82498142322857bec0c0977b3b519b3365bc4734454d902f58d4dbd50\": rpc error: code = NotFound desc = could not find container \"11f8b9a82498142322857bec0c0977b3b519b3365bc4734454d902f58d4dbd50\": container with ID starting with 11f8b9a82498142322857bec0c0977b3b519b3365bc4734454d902f58d4dbd50 not found: ID does not exist"
Sep 07 09:24:05.192443 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:05.192418    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"]
Sep 07 09:24:05.196405 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:05.196384    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/sub-test-model-34de95-kserve-7456489b8c-f4vd6"]
Sep 07 09:24:05.546050 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:05.545958    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c604300-7c74-4568-915f-de5b7151a94c" path="/var/lib/kubelet/pods/4c604300-7c74-4568-915f-de5b7151a94c/volumes"
Sep 07 09:24:05.546413 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:05.546395    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f15ee100-c58a-4ff6-85ae-3f52dd3ba762" path="/var/lib/kubelet/pods/f15ee100-c58a-4ff6-85ae-3f52dd3ba762/volumes"
Sep 07 09:24:09.716622 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:09.716587    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:11.497508 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:11.497474    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:14.801373 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:14.801340    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:24:17.392762 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.392729    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"]
Sep 07 09:24:17.393146 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.393120    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c604300-7c74-4568-915f-de5b7151a94c" containerName="main"
Sep 07 09:24:17.393146 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.393130    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c604300-7c74-4568-915f-de5b7151a94c" containerName="main"
Sep 07 09:24:17.393256 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.393164    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f15ee100-c58a-4ff6-85ae-3f52dd3ba762" containerName="main"
Sep 07 09:24:17.393256 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.393173    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f15ee100-c58a-4ff6-85ae-3f52dd3ba762" containerName="main"
Sep 07 09:24:17.393367 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.393318    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f15ee100-c58a-4ff6-85ae-3f52dd3ba762" containerName="main"
Sep 07 09:24:17.393367 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.393340    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c604300-7c74-4568-915f-de5b7151a94c" containerName="main"
Sep 07 09:24:17.396634 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.396615    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.399134 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.399110    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-3dc031\"/\"kube-root-ca.crt\""
Sep 07 09:24:17.399250 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.399154    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-3dc031\"/\"rate-test-model-3dc031-kserve-self-signed-certs\""
Sep 07 09:24:17.399250 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.399161    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-3dc031\"/\"openshift-service-ca.crt\""
Sep 07 09:24:17.399358 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.399262    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-3dc031\"/\"default-dockercfg-fm4xr\""
Sep 07 09:24:17.407103 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.406591    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"]
Sep 07 09:24:17.420384 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.420362    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:17.501736 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.501688    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-tmp-dir\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.501849 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.501752    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k26sx\" (UniqueName: \"kubernetes.io/projected/b0581d71-dd46-4520-bc40-4ed495464429-kube-api-access-k26sx\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.501916 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.501869    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-dshm\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.501916 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.501903    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0581d71-dd46-4520-bc40-4ed495464429-tls-certs\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.502019 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.501929    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-home\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.502077 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.502014    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-model-cache\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.602539 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.602517    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-dshm\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.602630 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.602543    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0581d71-dd46-4520-bc40-4ed495464429-tls-certs\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.602630 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.602559    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-home\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.602748 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.602691    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-model-cache\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.602809 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.602781    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-tmp-dir\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.602862 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.602830    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k26sx\" (UniqueName: \"kubernetes.io/projected/b0581d71-dd46-4520-bc40-4ed495464429-kube-api-access-k26sx\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.602988 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.602962    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-home\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.603087 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.603046    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-model-cache\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.603160 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.603125    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-tmp-dir\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.604887 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.604869    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-dshm\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.605131 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.605114    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0581d71-dd46-4520-bc40-4ed495464429-tls-certs\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.610097 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.610076    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k26sx\" (UniqueName: \"kubernetes.io/projected/b0581d71-dd46-4520-bc40-4ed495464429-kube-api-access-k26sx\") pod \"rate-test-model-3dc031-kserve-59d9995659-pz6f8\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") " pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.713950 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.713930    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:17.983536 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:17.983515    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"]
Sep 07 09:24:17.985633 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:24:17.985600    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0581d71_dd46_4520_bc40_4ed495464429.slice/crio-da68637be34e87a29f00c0e9ec204175f0f2c05460b0ea8621b717a5f4d5df1a WatchSource:0}: Error finding container da68637be34e87a29f00c0e9ec204175f0f2c05460b0ea8621b717a5f4d5df1a: Status 404 returned error can't find the container with id da68637be34e87a29f00c0e9ec204175f0f2c05460b0ea8621b717a5f4d5df1a
Sep 07 09:24:18.244116 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:18.244031    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8" event={"ID":"b0581d71-dd46-4520-bc40-4ed495464429","Type":"ContainerStarted","Data":"b088abd82845534d9a1e4d10df6664894faacf3517cbe91e0887f012852d7373"}
Sep 07 09:24:18.244116 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:18.244073    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8" event={"ID":"b0581d71-dd46-4520-bc40-4ed495464429","Type":"ContainerStarted","Data":"da68637be34e87a29f00c0e9ec204175f0f2c05460b0ea8621b717a5f4d5df1a"}
Sep 07 09:24:18.258294 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:18.258241    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8" podStartSLOduration=1.258223266 podStartE2EDuration="1.258223266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:24:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:24:18.258021303 +0000 UTC m=+1541.221937548" watchObservedRunningTime="2026-09-07 09:24:18.258223266 +0000 UTC m=+1541.222139486"
Sep 07 09:24:23.360292 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:23.360260    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:23.953534 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:23.953500    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:25.651594 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:25.651560    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"]
Sep 07 09:24:25.652059 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:25.651802    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9" podUID="834e08cf-4e50-4cf3-958b-673f1ce53a34" containerName="maas-api" containerID="cri-o://f9c386523c304842996ece01852b6e6c71f35a4b6b05494c378f19693553d263" gracePeriod=30
Sep 07 09:24:25.677060 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:25.677032    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:25.777834 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:25.777804    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:25.844578 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:25.844545    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:25.894067 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:25.894007    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:25.910876 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:25.910854    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:24:26.073272 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.073244    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-maas-api-tls\") pod \"834e08cf-4e50-4cf3-958b-673f1ce53a34\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") "
Sep 07 09:24:26.073434 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.073286    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/834e08cf-4e50-4cf3-958b-673f1ce53a34-openshift-service-ca\") pod \"834e08cf-4e50-4cf3-958b-673f1ce53a34\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") "
Sep 07 09:24:26.073434 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.073307    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxrt4\" (UniqueName: \"kubernetes.io/projected/834e08cf-4e50-4cf3-958b-673f1ce53a34-kube-api-access-cxrt4\") pod \"834e08cf-4e50-4cf3-958b-673f1ce53a34\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") "
Sep 07 09:24:26.073434 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.073394    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-metrics-certs\") pod \"834e08cf-4e50-4cf3-958b-673f1ce53a34\" (UID: \"834e08cf-4e50-4cf3-958b-673f1ce53a34\") "
Sep 07 09:24:26.073680 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.073648    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834e08cf-4e50-4cf3-958b-673f1ce53a34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "834e08cf-4e50-4cf3-958b-673f1ce53a34" (UID: "834e08cf-4e50-4cf3-958b-673f1ce53a34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:24:26.073781 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.073743    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/834e08cf-4e50-4cf3-958b-673f1ce53a34-openshift-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:26.075553 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.075520    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "834e08cf-4e50-4cf3-958b-673f1ce53a34" (UID: "834e08cf-4e50-4cf3-958b-673f1ce53a34"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:24:26.075671 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.075577    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/834e08cf-4e50-4cf3-958b-673f1ce53a34-kube-api-access-cxrt4" (OuterVolumeSpecName: "kube-api-access-cxrt4") pod "834e08cf-4e50-4cf3-958b-673f1ce53a34" (UID: "834e08cf-4e50-4cf3-958b-673f1ce53a34"). InnerVolumeSpecName "kube-api-access-cxrt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:24:26.075671 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.075578    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "834e08cf-4e50-4cf3-958b-673f1ce53a34" (UID: "834e08cf-4e50-4cf3-958b-673f1ce53a34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:24:26.175269 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.175180    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-maas-api-tls\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:26.175269 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.175213    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxrt4\" (UniqueName: \"kubernetes.io/projected/834e08cf-4e50-4cf3-958b-673f1ce53a34-kube-api-access-cxrt4\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:26.175269 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.175226    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/834e08cf-4e50-4cf3-958b-673f1ce53a34-metrics-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:24:26.191293 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.191263    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:26.288756 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.288698    2574 generic.go:356] "Generic (PLEG): container finished" podID="834e08cf-4e50-4cf3-958b-673f1ce53a34" containerID="f9c386523c304842996ece01852b6e6c71f35a4b6b05494c378f19693553d263" exitCode=0
Sep 07 09:24:26.288756 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.288744    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9" event={"ID":"834e08cf-4e50-4cf3-958b-673f1ce53a34","Type":"ContainerDied","Data":"f9c386523c304842996ece01852b6e6c71f35a4b6b05494c378f19693553d263"}
Sep 07 09:24:26.288756 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.288756    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"
Sep 07 09:24:26.289057 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.288801    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9" event={"ID":"834e08cf-4e50-4cf3-958b-673f1ce53a34","Type":"ContainerDied","Data":"ba569a4eded592c766f66b109efbc16fea124e823d078059591c95131301e712"}
Sep 07 09:24:26.289057 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.288827    2574 scope.go:117] "RemoveContainer" containerID="f9c386523c304842996ece01852b6e6c71f35a4b6b05494c378f19693553d263"
Sep 07 09:24:26.298972 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.298956    2574 scope.go:117] "RemoveContainer" containerID="f9c386523c304842996ece01852b6e6c71f35a4b6b05494c378f19693553d263"
Sep 07 09:24:26.299222 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:24:26.299203    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9c386523c304842996ece01852b6e6c71f35a4b6b05494c378f19693553d263\": container with ID starting with f9c386523c304842996ece01852b6e6c71f35a4b6b05494c378f19693553d263 not found: ID does not exist" containerID="f9c386523c304842996ece01852b6e6c71f35a4b6b05494c378f19693553d263"
Sep 07 09:24:26.299275 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.299231    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9c386523c304842996ece01852b6e6c71f35a4b6b05494c378f19693553d263"} err="failed to get container status \"f9c386523c304842996ece01852b6e6c71f35a4b6b05494c378f19693553d263\": rpc error: code = NotFound desc = could not find container \"f9c386523c304842996ece01852b6e6c71f35a4b6b05494c378f19693553d263\": container with ID starting with f9c386523c304842996ece01852b6e6c71f35a4b6b05494c378f19693553d263 not found: ID does not exist"
Sep 07 09:24:26.309455 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.309436    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"]
Sep 07 09:24:26.312681 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:26.312660    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-558343-77d575bb59-md4b9"]
Sep 07 09:24:27.551703 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:27.551668    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="834e08cf-4e50-4cf3-958b-673f1ce53a34" path="/var/lib/kubelet/pods/834e08cf-4e50-4cf3-958b-673f1ce53a34/volumes"
Sep 07 09:24:27.715142 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:27.715077    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:27.715333 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:27.715162    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:27.727471 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:27.727438    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:28.312066 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:28.312041    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:24:30.002157 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:30.002119    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:24:31.886424 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:31.886377    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:31.986579 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:31.986552    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:32.200363 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:32.200337    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:32.713299 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:32.713268    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:35.432950 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:35.432915    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:36.354236 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:36.354193    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:36.409291 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:36.409235    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:40.297608 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.297576    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"]
Sep 07 09:24:40.298001 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.297986    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="834e08cf-4e50-4cf3-958b-673f1ce53a34" containerName="maas-api"
Sep 07 09:24:40.298001 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.298003    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="834e08cf-4e50-4cf3-958b-673f1ce53a34" containerName="maas-api"
Sep 07 09:24:40.298097 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.298088    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="834e08cf-4e50-4cf3-958b-673f1ce53a34" containerName="maas-api"
Sep 07 09:24:40.302650 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.302630    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.305533 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.305510    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-34de95\"/\"kube-root-ca.crt\""
Sep 07 09:24:40.305657 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.305536    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-34de95\"/\"openshift-service-ca.crt\""
Sep 07 09:24:40.305657 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.305514    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-34de95\"/\"default-dockercfg-mflfq\""
Sep 07 09:24:40.305657 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.305519    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-34de95\"/\"rate-test-model-34de95-kserve-self-signed-certs\""
Sep 07 09:24:40.310870 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.310814    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"]
Sep 07 09:24:40.329627 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.329603    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:40.399978 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.399948    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-dshm\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.400125 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.399984    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-tmp-dir\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.400125 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.400013    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-home\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.400125 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.400066    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd659\" (UniqueName: \"kubernetes.io/projected/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-kube-api-access-bd659\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.400125 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.400099    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-tls-certs\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.400320 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.400208    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-model-cache\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.500791 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.500697    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-tls-certs\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.500916 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.500828    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-model-cache\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.500916 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.500893    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-dshm\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.501029 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.500930    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-tmp-dir\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.501029 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.500961    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-home\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.501029 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.500996    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd659\" (UniqueName: \"kubernetes.io/projected/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-kube-api-access-bd659\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.501270 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.501248    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-model-cache\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.501329 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.501299    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-home\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.501381 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.501364    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-tmp-dir\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.503278 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.503257    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-dshm\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.503378 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.503297    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-tls-certs\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.509421 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.509394    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd659\" (UniqueName: \"kubernetes.io/projected/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-kube-api-access-bd659\") pod \"rate-test-model-34de95-kserve-59ffbfc7f5-8flc5\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") " pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.613026 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.612995    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:40.888921 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:40.888891    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"]
Sep 07 09:24:40.889767 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:24:40.889747    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode21a10c0_96ae_43d9_b80f_812f1ba6d9d6.slice/crio-29bef0dd4e7bb768497aa62e28019d1d4775b506e45eef4a02edf55a8e8ffdf7 WatchSource:0}: Error finding container 29bef0dd4e7bb768497aa62e28019d1d4775b506e45eef4a02edf55a8e8ffdf7: Status 404 returned error can't find the container with id 29bef0dd4e7bb768497aa62e28019d1d4775b506e45eef4a02edf55a8e8ffdf7
Sep 07 09:24:41.371084 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:41.371056    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5" event={"ID":"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6","Type":"ContainerStarted","Data":"5f31c47377c6043c8504e84972221f1d53ccd697f1b010d7334832ef32132322"}
Sep 07 09:24:41.371437 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:41.371092    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5" event={"ID":"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6","Type":"ContainerStarted","Data":"29bef0dd4e7bb768497aa62e28019d1d4775b506e45eef4a02edf55a8e8ffdf7"}
Sep 07 09:24:41.386809 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:41.386762    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5" podStartSLOduration=1.386748051 podStartE2EDuration="1.386748051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:24:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:24:41.385928805 +0000 UTC m=+1564.349845044" watchObservedRunningTime="2026-09-07 09:24:41.386748051 +0000 UTC m=+1564.350664272"
Sep 07 09:24:42.852335 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:42.852303    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:42.917948 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:42.917905    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:42.978427 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:42.978399    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:43.039110 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:43.039080    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:43.400780 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:43.400751    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:43.706301 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:43.706265    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:24:49.006902 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:49.006865    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:49.107457 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:49.107424    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:49.256655 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:49.256619    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:49.830292 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:49.830258    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:24:50.613907 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:50.613881    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:50.614276 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:50.613926    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:50.625954 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:50.625932    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:24:51.429551 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:24:51.429517    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:25:00.047675 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.047636    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"]
Sep 07 09:25:00.048245 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.047879    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv" podUID="9c8d56a7-4b52-40ce-80f9-03ec699f1b3a" containerName="maas-api" containerID="cri-o://f61c38425a1158bfc7dfb52d8cf23da46cc551d160d748c2081fe8a2b2929014" gracePeriod=30
Sep 07 09:25:00.073959 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.073934    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:00.171773 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.171743    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:00.233112 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.233082    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:00.292953 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.292933    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:25:00.295041 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.295012    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:00.379641 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.379567    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-maas-api-tls\") pod \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") "
Sep 07 09:25:00.379800 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.379653    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-metrics-certs\") pod \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") "
Sep 07 09:25:00.379800 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.379687    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-openshift-service-ca\") pod \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") "
Sep 07 09:25:00.379931 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.379820    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5dcn\" (UniqueName: \"kubernetes.io/projected/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-kube-api-access-n5dcn\") pod \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\" (UID: \"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a\") "
Sep 07 09:25:00.380250 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.380215    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9c8d56a7-4b52-40ce-80f9-03ec699f1b3a" (UID: "9c8d56a7-4b52-40ce-80f9-03ec699f1b3a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:25:00.381917 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.381897    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9c8d56a7-4b52-40ce-80f9-03ec699f1b3a" (UID: "9c8d56a7-4b52-40ce-80f9-03ec699f1b3a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:25:00.382194 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.382175    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9c8d56a7-4b52-40ce-80f9-03ec699f1b3a" (UID: "9c8d56a7-4b52-40ce-80f9-03ec699f1b3a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:25:00.382459 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.382432    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-kube-api-access-n5dcn" (OuterVolumeSpecName: "kube-api-access-n5dcn") pod "9c8d56a7-4b52-40ce-80f9-03ec699f1b3a" (UID: "9c8d56a7-4b52-40ce-80f9-03ec699f1b3a"). InnerVolumeSpecName "kube-api-access-n5dcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:25:00.465234 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.465199    2574 generic.go:356] "Generic (PLEG): container finished" podID="9c8d56a7-4b52-40ce-80f9-03ec699f1b3a" containerID="f61c38425a1158bfc7dfb52d8cf23da46cc551d160d748c2081fe8a2b2929014" exitCode=0
Sep 07 09:25:00.465337 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.465241    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"
Sep 07 09:25:00.465337 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.465274    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv" event={"ID":"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a","Type":"ContainerDied","Data":"f61c38425a1158bfc7dfb52d8cf23da46cc551d160d748c2081fe8a2b2929014"}
Sep 07 09:25:00.465337 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.465318    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv" event={"ID":"9c8d56a7-4b52-40ce-80f9-03ec699f1b3a","Type":"ContainerDied","Data":"e57328a9ddd4ec5e667c5a877bac7dd720e9acea4cdfc63fcd4733625e7fb1d2"}
Sep 07 09:25:00.465453 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.465340    2574 scope.go:117] "RemoveContainer" containerID="f61c38425a1158bfc7dfb52d8cf23da46cc551d160d748c2081fe8a2b2929014"
Sep 07 09:25:00.474872 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.474858    2574 scope.go:117] "RemoveContainer" containerID="f61c38425a1158bfc7dfb52d8cf23da46cc551d160d748c2081fe8a2b2929014"
Sep 07 09:25:00.475138 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:25:00.475115    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f61c38425a1158bfc7dfb52d8cf23da46cc551d160d748c2081fe8a2b2929014\": container with ID starting with f61c38425a1158bfc7dfb52d8cf23da46cc551d160d748c2081fe8a2b2929014 not found: ID does not exist" containerID="f61c38425a1158bfc7dfb52d8cf23da46cc551d160d748c2081fe8a2b2929014"
Sep 07 09:25:00.475227 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.475140    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f61c38425a1158bfc7dfb52d8cf23da46cc551d160d748c2081fe8a2b2929014"} err="failed to get container status \"f61c38425a1158bfc7dfb52d8cf23da46cc551d160d748c2081fe8a2b2929014\": rpc error: code = NotFound desc = could not find container \"f61c38425a1158bfc7dfb52d8cf23da46cc551d160d748c2081fe8a2b2929014\": container with ID starting with f61c38425a1158bfc7dfb52d8cf23da46cc551d160d748c2081fe8a2b2929014 not found: ID does not exist"
Sep 07 09:25:00.481267 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.481249    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5dcn\" (UniqueName: \"kubernetes.io/projected/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-kube-api-access-n5dcn\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:00.481338 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.481269    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-maas-api-tls\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:00.481338 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.481277    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-metrics-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:00.481338 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.481285    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a-openshift-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:00.486091 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.486070    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"]
Sep 07 09:25:00.489741 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.489700    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5560ef-5dd67fc955-xzvwv"]
Sep 07 09:25:00.604155 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:00.604114    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:01.403183 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:01.403148    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:25:01.543072 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:01.543038    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c8d56a7-4b52-40ce-80f9-03ec699f1b3a" path="/var/lib/kubelet/pods/9c8d56a7-4b52-40ce-80f9-03ec699f1b3a/volumes"
Sep 07 09:25:06.254733 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:06.254673    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:06.318899 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:06.318872    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:06.501328 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:06.501298    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:11.297757 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:11.297727    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:25:20.495764 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:20.495728    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:25:26.719015 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:26.718984    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"]
Sep 07 09:25:26.737320 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:26.737291    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:27.677392 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:27.677366    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"]
Sep 07 09:25:27.695088 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:27.695058    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:28.111249 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:28.111177    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:28.406162 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:28.406132    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:28.518370 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:28.518304    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:30.791615 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:30.791585    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:25:31.429855 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:31.429817    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:31.523888 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:31.523862    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:32.747054 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:32.746971    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:32.869588 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:32.869563    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:32.964695 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:32.964663    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:33.195554 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:33.195521    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:33.297601 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:33.297565    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:41.739634 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:41.739588    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8" podUID="b0581d71-dd46-4520-bc40-4ed495464429" containerName="main" containerID="cri-o://b088abd82845534d9a1e4d10df6664894faacf3517cbe91e0887f012852d7373" gracePeriod=45
Sep 07 09:25:41.981194 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:41.981172    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:25:42.039412 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.039346    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-dshm\") pod \"b0581d71-dd46-4520-bc40-4ed495464429\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") "
Sep 07 09:25:42.039412 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.039409    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0581d71-dd46-4520-bc40-4ed495464429-tls-certs\") pod \"b0581d71-dd46-4520-bc40-4ed495464429\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") "
Sep 07 09:25:42.039564 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.039462    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k26sx\" (UniqueName: \"kubernetes.io/projected/b0581d71-dd46-4520-bc40-4ed495464429-kube-api-access-k26sx\") pod \"b0581d71-dd46-4520-bc40-4ed495464429\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") "
Sep 07 09:25:42.039564 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.039501    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-tmp-dir\") pod \"b0581d71-dd46-4520-bc40-4ed495464429\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") "
Sep 07 09:25:42.039564 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.039559    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-home\") pod \"b0581d71-dd46-4520-bc40-4ed495464429\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") "
Sep 07 09:25:42.039674 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.039585    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-model-cache\") pod \"b0581d71-dd46-4520-bc40-4ed495464429\" (UID: \"b0581d71-dd46-4520-bc40-4ed495464429\") "
Sep 07 09:25:42.039896 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.039839    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b0581d71-dd46-4520-bc40-4ed495464429" (UID: "b0581d71-dd46-4520-bc40-4ed495464429"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:25:42.040034 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.039969    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:42.040034 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.039977    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-model-cache" (OuterVolumeSpecName: "model-cache") pod "b0581d71-dd46-4520-bc40-4ed495464429" (UID: "b0581d71-dd46-4520-bc40-4ed495464429"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:25:42.040151 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.040035    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-home" (OuterVolumeSpecName: "home") pod "b0581d71-dd46-4520-bc40-4ed495464429" (UID: "b0581d71-dd46-4520-bc40-4ed495464429"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:25:42.041884 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.041861    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0581d71-dd46-4520-bc40-4ed495464429-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b0581d71-dd46-4520-bc40-4ed495464429" (UID: "b0581d71-dd46-4520-bc40-4ed495464429"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:25:42.042128 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.042110    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0581d71-dd46-4520-bc40-4ed495464429-kube-api-access-k26sx" (OuterVolumeSpecName: "kube-api-access-k26sx") pod "b0581d71-dd46-4520-bc40-4ed495464429" (UID: "b0581d71-dd46-4520-bc40-4ed495464429"). InnerVolumeSpecName "kube-api-access-k26sx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:25:42.042190 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.042172    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-dshm" (OuterVolumeSpecName: "dshm") pod "b0581d71-dd46-4520-bc40-4ed495464429" (UID: "b0581d71-dd46-4520-bc40-4ed495464429"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:25:42.140450 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.140411    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:42.140450 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.140446    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0581d71-dd46-4520-bc40-4ed495464429-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:42.140450 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.140456    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k26sx\" (UniqueName: \"kubernetes.io/projected/b0581d71-dd46-4520-bc40-4ed495464429-kube-api-access-k26sx\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:42.140636 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.140465    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:42.140636 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.140473    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0581d71-dd46-4520-bc40-4ed495464429-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:42.389995 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.389908    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:25:42.676821 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.676792    2574 generic.go:356] "Generic (PLEG): container finished" podID="b0581d71-dd46-4520-bc40-4ed495464429" containerID="b088abd82845534d9a1e4d10df6664894faacf3517cbe91e0887f012852d7373" exitCode=0
Sep 07 09:25:42.677008 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.676834    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"
Sep 07 09:25:42.677008 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.676880    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8" event={"ID":"b0581d71-dd46-4520-bc40-4ed495464429","Type":"ContainerDied","Data":"b088abd82845534d9a1e4d10df6664894faacf3517cbe91e0887f012852d7373"}
Sep 07 09:25:42.677008 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.676925    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8" event={"ID":"b0581d71-dd46-4520-bc40-4ed495464429","Type":"ContainerDied","Data":"da68637be34e87a29f00c0e9ec204175f0f2c05460b0ea8621b717a5f4d5df1a"}
Sep 07 09:25:42.677008 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.676947    2574 scope.go:117] "RemoveContainer" containerID="b088abd82845534d9a1e4d10df6664894faacf3517cbe91e0887f012852d7373"
Sep 07 09:25:42.686053 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.686036    2574 scope.go:117] "RemoveContainer" containerID="b088abd82845534d9a1e4d10df6664894faacf3517cbe91e0887f012852d7373"
Sep 07 09:25:42.686348 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:25:42.686331    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b088abd82845534d9a1e4d10df6664894faacf3517cbe91e0887f012852d7373\": container with ID starting with b088abd82845534d9a1e4d10df6664894faacf3517cbe91e0887f012852d7373 not found: ID does not exist" containerID="b088abd82845534d9a1e4d10df6664894faacf3517cbe91e0887f012852d7373"
Sep 07 09:25:42.686409 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.686358    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b088abd82845534d9a1e4d10df6664894faacf3517cbe91e0887f012852d7373"} err="failed to get container status \"b088abd82845534d9a1e4d10df6664894faacf3517cbe91e0887f012852d7373\": rpc error: code = NotFound desc = could not find container \"b088abd82845534d9a1e4d10df6664894faacf3517cbe91e0887f012852d7373\": container with ID starting with b088abd82845534d9a1e4d10df6664894faacf3517cbe91e0887f012852d7373 not found: ID does not exist"
Sep 07 09:25:42.696583 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.696560    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"]
Sep 07 09:25:42.698809 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.698789    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5" podUID="e21a10c0-96ae-43d9-b80f-812f1ba6d9d6" containerName="main" containerID="cri-o://5f31c47377c6043c8504e84972221f1d53ccd697f1b010d7334832ef32132322" gracePeriod=45
Sep 07 09:25:42.699678 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.699660    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3dc031/rate-test-model-3dc031-kserve-59d9995659-pz6f8"]
Sep 07 09:25:42.939555 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.939500    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:25:42.944559 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.944537    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bd659\" (UniqueName: \"kubernetes.io/projected/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-kube-api-access-bd659\") pod \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") "
Sep 07 09:25:42.944613 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.944580    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-model-cache\") pod \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") "
Sep 07 09:25:42.944645 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.944618    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-dshm\") pod \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") "
Sep 07 09:25:42.944689 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.944663    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-tmp-dir\") pod \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") "
Sep 07 09:25:42.944770 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.944743    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-tls-certs\") pod \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") "
Sep 07 09:25:42.944827 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.944786    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-home\") pod \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\" (UID: \"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6\") "
Sep 07 09:25:42.944918 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.944874    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-model-cache" (OuterVolumeSpecName: "model-cache") pod "e21a10c0-96ae-43d9-b80f-812f1ba6d9d6" (UID: "e21a10c0-96ae-43d9-b80f-812f1ba6d9d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:25:42.944979 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.944957    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e21a10c0-96ae-43d9-b80f-812f1ba6d9d6" (UID: "e21a10c0-96ae-43d9-b80f-812f1ba6d9d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:25:42.945066 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.945051    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:42.945125 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.945071    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:42.945125 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.945067    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-home" (OuterVolumeSpecName: "home") pod "e21a10c0-96ae-43d9-b80f-812f1ba6d9d6" (UID: "e21a10c0-96ae-43d9-b80f-812f1ba6d9d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:25:42.946795 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.946766    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-dshm" (OuterVolumeSpecName: "dshm") pod "e21a10c0-96ae-43d9-b80f-812f1ba6d9d6" (UID: "e21a10c0-96ae-43d9-b80f-812f1ba6d9d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:25:42.946901 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.946877    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-kube-api-access-bd659" (OuterVolumeSpecName: "kube-api-access-bd659") pod "e21a10c0-96ae-43d9-b80f-812f1ba6d9d6" (UID: "e21a10c0-96ae-43d9-b80f-812f1ba6d9d6"). InnerVolumeSpecName "kube-api-access-bd659". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:25:42.947132 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:42.947115    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e21a10c0-96ae-43d9-b80f-812f1ba6d9d6" (UID: "e21a10c0-96ae-43d9-b80f-812f1ba6d9d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:25:43.045755 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:43.045728    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd659\" (UniqueName: \"kubernetes.io/projected/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-kube-api-access-bd659\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:43.045755 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:43.045757    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:43.045902 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:43.045774    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:43.045902 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:43.045787    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:25:43.542682 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:43.542639    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0581d71-dd46-4520-bc40-4ed495464429" path="/var/lib/kubelet/pods/b0581d71-dd46-4520-bc40-4ed495464429/volumes"
Sep 07 09:25:43.683933 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:43.683903    2574 generic.go:356] "Generic (PLEG): container finished" podID="e21a10c0-96ae-43d9-b80f-812f1ba6d9d6" containerID="5f31c47377c6043c8504e84972221f1d53ccd697f1b010d7334832ef32132322" exitCode=0
Sep 07 09:25:43.684062 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:43.683949    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"
Sep 07 09:25:43.684062 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:43.683944    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5" event={"ID":"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6","Type":"ContainerDied","Data":"5f31c47377c6043c8504e84972221f1d53ccd697f1b010d7334832ef32132322"}
Sep 07 09:25:43.684062 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:43.684060    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5" event={"ID":"e21a10c0-96ae-43d9-b80f-812f1ba6d9d6","Type":"ContainerDied","Data":"29bef0dd4e7bb768497aa62e28019d1d4775b506e45eef4a02edf55a8e8ffdf7"}
Sep 07 09:25:43.684187 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:43.684077    2574 scope.go:117] "RemoveContainer" containerID="5f31c47377c6043c8504e84972221f1d53ccd697f1b010d7334832ef32132322"
Sep 07 09:25:43.692250 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:43.692233    2574 scope.go:117] "RemoveContainer" containerID="5f31c47377c6043c8504e84972221f1d53ccd697f1b010d7334832ef32132322"
Sep 07 09:25:43.692518 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:25:43.692501    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f31c47377c6043c8504e84972221f1d53ccd697f1b010d7334832ef32132322\": container with ID starting with 5f31c47377c6043c8504e84972221f1d53ccd697f1b010d7334832ef32132322 not found: ID does not exist" containerID="5f31c47377c6043c8504e84972221f1d53ccd697f1b010d7334832ef32132322"
Sep 07 09:25:43.692571 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:43.692526    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f31c47377c6043c8504e84972221f1d53ccd697f1b010d7334832ef32132322"} err="failed to get container status \"5f31c47377c6043c8504e84972221f1d53ccd697f1b010d7334832ef32132322\": rpc error: code = NotFound desc = could not find container \"5f31c47377c6043c8504e84972221f1d53ccd697f1b010d7334832ef32132322\": container with ID starting with 5f31c47377c6043c8504e84972221f1d53ccd697f1b010d7334832ef32132322 not found: ID does not exist"
Sep 07 09:25:43.697732 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:43.697688    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"]
Sep 07 09:25:43.702814 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:43.702795    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-34de95/rate-test-model-34de95-kserve-59ffbfc7f5-8flc5"]
Sep 07 09:25:45.542584 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:45.542549    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e21a10c0-96ae-43d9-b80f-812f1ba6d9d6" path="/var/lib/kubelet/pods/e21a10c0-96ae-43d9-b80f-812f1ba6d9d6/volumes"
Sep 07 09:25:56.488738 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:56.488678    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:57.197154 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:57.197119    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:57.520348 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:57.520270    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:25:58.042804 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:25:58.042767    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:26:00.658799 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:00.658767    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:26:00.754839 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:00.754809    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:26:01.754298 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.754266    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"]
Sep 07 09:26:01.754875 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.754845    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0581d71-dd46-4520-bc40-4ed495464429" containerName="main"
Sep 07 09:26:01.754875 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.754866    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0581d71-dd46-4520-bc40-4ed495464429" containerName="main"
Sep 07 09:26:01.755063 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.754894    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c8d56a7-4b52-40ce-80f9-03ec699f1b3a" containerName="maas-api"
Sep 07 09:26:01.755063 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.754903    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c8d56a7-4b52-40ce-80f9-03ec699f1b3a" containerName="maas-api"
Sep 07 09:26:01.755063 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.754912    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e21a10c0-96ae-43d9-b80f-812f1ba6d9d6" containerName="main"
Sep 07 09:26:01.755063 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.754920    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e21a10c0-96ae-43d9-b80f-812f1ba6d9d6" containerName="main"
Sep 07 09:26:01.755063 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.755017    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e21a10c0-96ae-43d9-b80f-812f1ba6d9d6" containerName="main"
Sep 07 09:26:01.755063 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.755030    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c8d56a7-4b52-40ce-80f9-03ec699f1b3a" containerName="maas-api"
Sep 07 09:26:01.755063 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.755038    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0581d71-dd46-4520-bc40-4ed495464429" containerName="main"
Sep 07 09:26:01.758401 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.758380    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:01.760454 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.760425    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-da0a32-serving-cert\""
Sep 07 09:26:01.766647 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.766051    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"]
Sep 07 09:26:01.783994 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.783966    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:26:01.794148 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.794128    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a39d117-58c6-45f4-8c7a-909e886f0027-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:01.794261 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.794153    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a39d117-58c6-45f4-8c7a-909e886f0027-maas-api-tls\") pod \"maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:01.794261 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.794169    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a39d117-58c6-45f4-8c7a-909e886f0027-metrics-certs\") pod \"maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:01.794261 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.794196    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdhwl\" (UniqueName: \"kubernetes.io/projected/6a39d117-58c6-45f4-8c7a-909e886f0027-kube-api-access-xdhwl\") pod \"maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:01.884955 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.884919    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:26:01.895202 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.895186    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a39d117-58c6-45f4-8c7a-909e886f0027-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:01.895304 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.895210    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a39d117-58c6-45f4-8c7a-909e886f0027-maas-api-tls\") pod \"maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:01.895304 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.895228    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a39d117-58c6-45f4-8c7a-909e886f0027-metrics-certs\") pod \"maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:01.895304 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.895249    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdhwl\" (UniqueName: \"kubernetes.io/projected/6a39d117-58c6-45f4-8c7a-909e886f0027-kube-api-access-xdhwl\") pod \"maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:01.895817 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.895793    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a39d117-58c6-45f4-8c7a-909e886f0027-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:01.897933 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.897910    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a39d117-58c6-45f4-8c7a-909e886f0027-maas-api-tls\") pod \"maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:01.898038 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.897932    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a39d117-58c6-45f4-8c7a-909e886f0027-metrics-certs\") pod \"maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:01.903348 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.903325    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdhwl\" (UniqueName: \"kubernetes.io/projected/6a39d117-58c6-45f4-8c7a-909e886f0027-kube-api-access-xdhwl\") pod \"maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:01.984837 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:01.984809    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:26:02.072279 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:02.072193    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:02.260157 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:02.260126    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:26:02.345088 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:02.345065    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"]
Sep 07 09:26:02.347564 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:26:02.347538    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a39d117_58c6_45f4_8c7a_909e886f0027.slice/crio-033bbb7107b44660f2040676f2a2da820bd85d00f9b1945d364d7b1c009d2ed1 WatchSource:0}: Error finding container 033bbb7107b44660f2040676f2a2da820bd85d00f9b1945d364d7b1c009d2ed1: Status 404 returned error can't find the container with id 033bbb7107b44660f2040676f2a2da820bd85d00f9b1945d364d7b1c009d2ed1
Sep 07 09:26:02.348657 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:02.348623    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:26:02.349162 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:02.349146    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 09:26:02.780377 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:02.780344    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p" event={"ID":"6a39d117-58c6-45f4-8c7a-909e886f0027","Type":"ContainerStarted","Data":"033bbb7107b44660f2040676f2a2da820bd85d00f9b1945d364d7b1c009d2ed1"}
Sep 07 09:26:03.786530 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:03.786494    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p" event={"ID":"6a39d117-58c6-45f4-8c7a-909e886f0027","Type":"ContainerStarted","Data":"92788330236cfe8201da0b261164b2763b961454f404a60fc18139d263ac0a3f"}
Sep 07 09:26:03.786992 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:03.786633    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:03.801791 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:03.801750    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p" podStartSLOduration=2.304657122 podStartE2EDuration="2.801736334s" totalImagesPullingTime="497.079212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:26:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:26:03.799866163 +0000 UTC m=+1646.763782382" watchObservedRunningTime="2026-09-07 09:26:03.801736334 +0000 UTC m=+1646.765652566"
Sep 07 09:26:05.503451 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:05.503415    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:26:09.798634 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:09.798606    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:26:19.879502 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:19.879463    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:26:19.929817 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:19.929698    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:26:25.636381 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:25.636347    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:26:42.097245 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.097213    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"]
Sep 07 09:26:42.101994 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.101974    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.104733 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.104696    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-0a9a72\"/\"openshift-service-ca.crt\""
Sep 07 09:26:42.104939 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.104889    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-0a9a72\"/\"default-dockercfg-6gtmh\""
Sep 07 09:26:42.105045 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.104836    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-0a9a72\"/\"ipp-route-0a9a72-kserve-self-signed-certs\""
Sep 07 09:26:42.105187 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.104769    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-0a9a72\"/\"kube-root-ca.crt\""
Sep 07 09:26:42.109698 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.109675    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"]
Sep 07 09:26:42.127344 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.127317    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:26:42.134672 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.134624    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hcsq\" (UniqueName: \"kubernetes.io/projected/6bd75921-f1b0-41ba-9780-8ed6850919f8-kube-api-access-7hcsq\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.134794 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.134685    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-home\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.134794 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.134741    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-tmp-dir\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.134794 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.134764    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-dshm\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.134958 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.134796    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bd75921-f1b0-41ba-9780-8ed6850919f8-tls-certs\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.134958 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.134936    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-model-cache\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.236215 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.236187    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-home\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.236368 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.236231    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-tmp-dir\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.236368 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.236247    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-dshm\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.236368 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.236275    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bd75921-f1b0-41ba-9780-8ed6850919f8-tls-certs\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.236368 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.236361    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-model-cache\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.236543 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.236412    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hcsq\" (UniqueName: \"kubernetes.io/projected/6bd75921-f1b0-41ba-9780-8ed6850919f8-kube-api-access-7hcsq\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.236669 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.236626    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-home\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.236804 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.236763    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-tmp-dir\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.236804 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.236782    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-model-cache\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.238790 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.238767    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-dshm\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.238911 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.238895    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bd75921-f1b0-41ba-9780-8ed6850919f8-tls-certs\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.244112 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.244079    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hcsq\" (UniqueName: \"kubernetes.io/projected/6bd75921-f1b0-41ba-9780-8ed6850919f8-kube-api-access-7hcsq\") pod \"ipp-route-0a9a72-kserve-7fc48cb859-4s9ww\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") " pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.412888 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.412861    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:42.687518 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.687494    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"]
Sep 07 09:26:42.689515 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:26:42.689495    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bd75921_f1b0_41ba_9780_8ed6850919f8.slice/crio-67f13fe107180d5f2b7fd1b42790a5f2e164b6d6fb465cdb1aab2cd1e4b1f663 WatchSource:0}: Error finding container 67f13fe107180d5f2b7fd1b42790a5f2e164b6d6fb465cdb1aab2cd1e4b1f663: Status 404 returned error can't find the container with id 67f13fe107180d5f2b7fd1b42790a5f2e164b6d6fb465cdb1aab2cd1e4b1f663
Sep 07 09:26:42.992995 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.992916    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww" event={"ID":"6bd75921-f1b0-41ba-9780-8ed6850919f8","Type":"ContainerStarted","Data":"0d441c89a4df0f87e2ce085849cd18f0f2a8b4c53b118d2d76033ad9b4f54753"}
Sep 07 09:26:42.992995 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:42.992953    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww" event={"ID":"6bd75921-f1b0-41ba-9780-8ed6850919f8","Type":"ContainerStarted","Data":"67f13fe107180d5f2b7fd1b42790a5f2e164b6d6fb465cdb1aab2cd1e4b1f663"}
Sep 07 09:26:43.009107 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:43.009063    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww" podStartSLOduration=1.009047682 podStartE2EDuration="1.009047682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:26:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:26:43.007827658 +0000 UTC m=+1685.971743897" watchObservedRunningTime="2026-09-07 09:26:43.009047682 +0000 UTC m=+1685.972963901"
Sep 07 09:26:47.298339 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:47.298305    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:26:52.413956 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:52.413916    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:52.414343 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:52.413987    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:52.426205 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:52.426175    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:26:53.055278 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:26:53.055252    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:27:02.596867 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:02.596791    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:27:08.939405 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:08.939370    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:09.230515 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:09.230426    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:09.564704 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:09.564618    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:11.981110 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:11.981070    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:12.090848 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:12.090819    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:13.084088 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.084058    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"]
Sep 07 09:27:13.087698 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.087677    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:13.090555 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.090534    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-d9283c-serving-cert\""
Sep 07 09:27:13.098395 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.098370    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"]
Sep 07 09:27:13.121236 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.121212    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:13.206092 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.206060    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nxtc\" (UniqueName: \"kubernetes.io/projected/6f7b84ef-f29a-456c-83ff-631ab822b2b6-kube-api-access-7nxtc\") pod \"maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:13.206237 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.206124    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f7b84ef-f29a-456c-83ff-631ab822b2b6-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:13.206237 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.206181    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f7b84ef-f29a-456c-83ff-631ab822b2b6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:13.206355 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.206326    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f7b84ef-f29a-456c-83ff-631ab822b2b6-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:13.222863 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.222837    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:13.307032 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.307002    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f7b84ef-f29a-456c-83ff-631ab822b2b6-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:13.307178 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.307061    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nxtc\" (UniqueName: \"kubernetes.io/projected/6f7b84ef-f29a-456c-83ff-631ab822b2b6-kube-api-access-7nxtc\") pod \"maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:13.307178 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.307111    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f7b84ef-f29a-456c-83ff-631ab822b2b6-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:13.307178 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.307137    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f7b84ef-f29a-456c-83ff-631ab822b2b6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:13.307823 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.307803    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f7b84ef-f29a-456c-83ff-631ab822b2b6-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:13.309451 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.309418    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f7b84ef-f29a-456c-83ff-631ab822b2b6-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:13.309602 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.309583    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f7b84ef-f29a-456c-83ff-631ab822b2b6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:13.314879 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.314863    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nxtc\" (UniqueName: \"kubernetes.io/projected/6f7b84ef-f29a-456c-83ff-631ab822b2b6-kube-api-access-7nxtc\") pod \"maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:13.328869 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.328849    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:13.400499 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.400450    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:13.589771 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.589740    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:13.671773 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.671746    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"]
Sep 07 09:27:13.674260 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:13.674209    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:13.674394 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:27:13.674291    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f7b84ef_f29a_456c_83ff_631ab822b2b6.slice/crio-58ef8be1d897a3a836d2c057cc55d08816a727ae3ce59037e9cd971ab89b5d63 WatchSource:0}: Error finding container 58ef8be1d897a3a836d2c057cc55d08816a727ae3ce59037e9cd971ab89b5d63: Status 404 returned error can't find the container with id 58ef8be1d897a3a836d2c057cc55d08816a727ae3ce59037e9cd971ab89b5d63
Sep 07 09:27:14.155201 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:14.155172    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf" event={"ID":"6f7b84ef-f29a-456c-83ff-631ab822b2b6","Type":"ContainerStarted","Data":"59a6537a9002befb59cbd9708ece3f116711a0a3be28d5d83024218cb66d910c"}
Sep 07 09:27:14.155596 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:14.155218    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf" event={"ID":"6f7b84ef-f29a-456c-83ff-631ab822b2b6","Type":"ContainerStarted","Data":"58ef8be1d897a3a836d2c057cc55d08816a727ae3ce59037e9cd971ab89b5d63"}
Sep 07 09:27:14.155596 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:14.155328    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:14.173112 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:14.173064    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf" podStartSLOduration=0.847384929 podStartE2EDuration="1.17304816s" totalImagesPullingTime="325.663231ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:27:14.171648487 +0000 UTC m=+1717.135564740" watchObservedRunningTime="2026-09-07 09:27:14.17304816 +0000 UTC m=+1717.136964379"
Sep 07 09:27:18.507647 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:18.507614    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:20.165830 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:20.165802    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:21.998893 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:21.998844    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:27:36.951741 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:36.951692    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:43.518530 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.518493    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"]
Sep 07 09:27:43.519090 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.518792    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf" podUID="6f7b84ef-f29a-456c-83ff-631ab822b2b6" containerName="maas-api" containerID="cri-o://59a6537a9002befb59cbd9708ece3f116711a0a3be28d5d83024218cb66d910c" gracePeriod=30
Sep 07 09:27:43.546933 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.546901    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:43.647267 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.647231    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:43.747638 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.747606    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:43.777749 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.777680    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:43.814694 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.814667    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:43.860837 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.860814    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nxtc\" (UniqueName: \"kubernetes.io/projected/6f7b84ef-f29a-456c-83ff-631ab822b2b6-kube-api-access-7nxtc\") pod \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") "
Sep 07 09:27:43.860945 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.860857    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f7b84ef-f29a-456c-83ff-631ab822b2b6-openshift-service-ca\") pod \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") "
Sep 07 09:27:43.861002 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.860965    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f7b84ef-f29a-456c-83ff-631ab822b2b6-maas-api-tls\") pod \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") "
Sep 07 09:27:43.861002 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.860989    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f7b84ef-f29a-456c-83ff-631ab822b2b6-metrics-certs\") pod \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\" (UID: \"6f7b84ef-f29a-456c-83ff-631ab822b2b6\") "
Sep 07 09:27:43.861287 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.861260    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f7b84ef-f29a-456c-83ff-631ab822b2b6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f7b84ef-f29a-456c-83ff-631ab822b2b6" (UID: "6f7b84ef-f29a-456c-83ff-631ab822b2b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:27:43.863159 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.863137    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f7b84ef-f29a-456c-83ff-631ab822b2b6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f7b84ef-f29a-456c-83ff-631ab822b2b6" (UID: "6f7b84ef-f29a-456c-83ff-631ab822b2b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:27:43.863262 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.863234    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f7b84ef-f29a-456c-83ff-631ab822b2b6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6f7b84ef-f29a-456c-83ff-631ab822b2b6" (UID: "6f7b84ef-f29a-456c-83ff-631ab822b2b6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:27:43.863315 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.863302    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7b84ef-f29a-456c-83ff-631ab822b2b6-kube-api-access-7nxtc" (OuterVolumeSpecName: "kube-api-access-7nxtc") pod "6f7b84ef-f29a-456c-83ff-631ab822b2b6" (UID: "6f7b84ef-f29a-456c-83ff-631ab822b2b6"). InnerVolumeSpecName "kube-api-access-7nxtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:27:43.961958 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.961928    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nxtc\" (UniqueName: \"kubernetes.io/projected/6f7b84ef-f29a-456c-83ff-631ab822b2b6-kube-api-access-7nxtc\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:27:43.961958 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.961955    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f7b84ef-f29a-456c-83ff-631ab822b2b6-openshift-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:27:43.961958 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.961964    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f7b84ef-f29a-456c-83ff-631ab822b2b6-maas-api-tls\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:27:43.962168 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:43.961972    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f7b84ef-f29a-456c-83ff-631ab822b2b6-metrics-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:27:44.145181 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:44.145076    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:44.302724 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:44.302681    2574 generic.go:356] "Generic (PLEG): container finished" podID="6f7b84ef-f29a-456c-83ff-631ab822b2b6" containerID="59a6537a9002befb59cbd9708ece3f116711a0a3be28d5d83024218cb66d910c" exitCode=0
Sep 07 09:27:44.302884 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:44.302748    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"
Sep 07 09:27:44.302884 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:44.302760    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf" event={"ID":"6f7b84ef-f29a-456c-83ff-631ab822b2b6","Type":"ContainerDied","Data":"59a6537a9002befb59cbd9708ece3f116711a0a3be28d5d83024218cb66d910c"}
Sep 07 09:27:44.302884 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:44.302812    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf" event={"ID":"6f7b84ef-f29a-456c-83ff-631ab822b2b6","Type":"ContainerDied","Data":"58ef8be1d897a3a836d2c057cc55d08816a727ae3ce59037e9cd971ab89b5d63"}
Sep 07 09:27:44.302884 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:44.302842    2574 scope.go:117] "RemoveContainer" containerID="59a6537a9002befb59cbd9708ece3f116711a0a3be28d5d83024218cb66d910c"
Sep 07 09:27:44.312439 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:44.312421    2574 scope.go:117] "RemoveContainer" containerID="59a6537a9002befb59cbd9708ece3f116711a0a3be28d5d83024218cb66d910c"
Sep 07 09:27:44.312730 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:27:44.312696    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59a6537a9002befb59cbd9708ece3f116711a0a3be28d5d83024218cb66d910c\": container with ID starting with 59a6537a9002befb59cbd9708ece3f116711a0a3be28d5d83024218cb66d910c not found: ID does not exist" containerID="59a6537a9002befb59cbd9708ece3f116711a0a3be28d5d83024218cb66d910c"
Sep 07 09:27:44.312785 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:44.312740    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59a6537a9002befb59cbd9708ece3f116711a0a3be28d5d83024218cb66d910c"} err="failed to get container status \"59a6537a9002befb59cbd9708ece3f116711a0a3be28d5d83024218cb66d910c\": rpc error: code = NotFound desc = could not find container \"59a6537a9002befb59cbd9708ece3f116711a0a3be28d5d83024218cb66d910c\": container with ID starting with 59a6537a9002befb59cbd9708ece3f116711a0a3be28d5d83024218cb66d910c not found: ID does not exist"
Sep 07 09:27:44.322606 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:44.322577    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"]
Sep 07 09:27:44.324508 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:44.324489    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d9283c-85957f6dd9-9h6hf"]
Sep 07 09:27:45.544895 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:45.544862    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f7b84ef-f29a-456c-83ff-631ab822b2b6" path="/var/lib/kubelet/pods/6f7b84ef-f29a-456c-83ff-631ab822b2b6/volumes"
Sep 07 09:27:49.708313 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:49.708265    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:49.798160 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:49.798129    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:27:49.800866 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:49.800836    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:49.975060 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:49.974974    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:27:53.449587 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:27:53.449551    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:28:08.096869 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:08.096832    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:28:08.815324 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:08.815293    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:28:08.881551 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:08.881517    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:28:08.983321 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:08.983287    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:28:09.062448 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:09.062409    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:28:09.363288 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:09.363257    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:28:10.151137 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:10.151099    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:28:14.339979 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:14.339937    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"]
Sep 07 09:28:14.436626 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:14.436593    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:28:18.004394 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:18.004354    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:28:29.360905 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.360819    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww" podUID="6bd75921-f1b0-41ba-9780-8ed6850919f8" containerName="main" containerID="cri-o://0d441c89a4df0f87e2ce085849cd18f0f2a8b4c53b118d2d76033ad9b4f54753" gracePeriod=45
Sep 07 09:28:29.542634 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.542588    2574 generic.go:356] "Generic (PLEG): container finished" podID="6bd75921-f1b0-41ba-9780-8ed6850919f8" containerID="0d441c89a4df0f87e2ce085849cd18f0f2a8b4c53b118d2d76033ad9b4f54753" exitCode=0
Sep 07 09:28:29.548945 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.548917    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww" event={"ID":"6bd75921-f1b0-41ba-9780-8ed6850919f8","Type":"ContainerDied","Data":"0d441c89a4df0f87e2ce085849cd18f0f2a8b4c53b118d2d76033ad9b4f54753"}
Sep 07 09:28:29.615507 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.615454    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:28:29.754498 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.754466    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-tmp-dir\") pod \"6bd75921-f1b0-41ba-9780-8ed6850919f8\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") "
Sep 07 09:28:29.754498 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.754499    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-model-cache\") pod \"6bd75921-f1b0-41ba-9780-8ed6850919f8\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") "
Sep 07 09:28:29.754747 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.754557    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-dshm\") pod \"6bd75921-f1b0-41ba-9780-8ed6850919f8\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") "
Sep 07 09:28:29.754747 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.754598    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hcsq\" (UniqueName: \"kubernetes.io/projected/6bd75921-f1b0-41ba-9780-8ed6850919f8-kube-api-access-7hcsq\") pod \"6bd75921-f1b0-41ba-9780-8ed6850919f8\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") "
Sep 07 09:28:29.754747 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.754621    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bd75921-f1b0-41ba-9780-8ed6850919f8-tls-certs\") pod \"6bd75921-f1b0-41ba-9780-8ed6850919f8\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") "
Sep 07 09:28:29.754747 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.754702    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-home\") pod \"6bd75921-f1b0-41ba-9780-8ed6850919f8\" (UID: \"6bd75921-f1b0-41ba-9780-8ed6850919f8\") "
Sep 07 09:28:29.754945 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.754844    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6bd75921-f1b0-41ba-9780-8ed6850919f8" (UID: "6bd75921-f1b0-41ba-9780-8ed6850919f8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:28:29.754945 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.754873    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-model-cache" (OuterVolumeSpecName: "model-cache") pod "6bd75921-f1b0-41ba-9780-8ed6850919f8" (UID: "6bd75921-f1b0-41ba-9780-8ed6850919f8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:28:29.755100 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.755071    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:28:29.755100 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.755095    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:28:29.755293 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.755144    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-home" (OuterVolumeSpecName: "home") pod "6bd75921-f1b0-41ba-9780-8ed6850919f8" (UID: "6bd75921-f1b0-41ba-9780-8ed6850919f8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:28:29.757635 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.757598    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bd75921-f1b0-41ba-9780-8ed6850919f8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6bd75921-f1b0-41ba-9780-8ed6850919f8" (UID: "6bd75921-f1b0-41ba-9780-8ed6850919f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:28:29.757760 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.757704    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bd75921-f1b0-41ba-9780-8ed6850919f8-kube-api-access-7hcsq" (OuterVolumeSpecName: "kube-api-access-7hcsq") pod "6bd75921-f1b0-41ba-9780-8ed6850919f8" (UID: "6bd75921-f1b0-41ba-9780-8ed6850919f8"). InnerVolumeSpecName "kube-api-access-7hcsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:28:29.757829 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.757768    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-dshm" (OuterVolumeSpecName: "dshm") pod "6bd75921-f1b0-41ba-9780-8ed6850919f8" (UID: "6bd75921-f1b0-41ba-9780-8ed6850919f8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:28:29.855977 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.855948    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:28:29.855977 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.855972    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hcsq\" (UniqueName: \"kubernetes.io/projected/6bd75921-f1b0-41ba-9780-8ed6850919f8-kube-api-access-7hcsq\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:28:29.855977 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.855982    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bd75921-f1b0-41ba-9780-8ed6850919f8-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:28:29.856170 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:29.855991    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bd75921-f1b0-41ba-9780-8ed6850919f8-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:28:30.549415 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:30.549384    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww" event={"ID":"6bd75921-f1b0-41ba-9780-8ed6850919f8","Type":"ContainerDied","Data":"67f13fe107180d5f2b7fd1b42790a5f2e164b6d6fb465cdb1aab2cd1e4b1f663"}
Sep 07 09:28:30.549415 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:30.549412    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"
Sep 07 09:28:30.549957 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:30.549443    2574 scope.go:117] "RemoveContainer" containerID="0d441c89a4df0f87e2ce085849cd18f0f2a8b4c53b118d2d76033ad9b4f54753"
Sep 07 09:28:30.570864 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:30.570841    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"]
Sep 07 09:28:30.576087 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:30.576067    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-0a9a72/ipp-route-0a9a72-kserve-7fc48cb859-4s9ww"]
Sep 07 09:28:31.542613 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:31.542571    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bd75921-f1b0-41ba-9780-8ed6850919f8" path="/var/lib/kubelet/pods/6bd75921-f1b0-41ba-9780-8ed6850919f8/volumes"
Sep 07 09:28:37.606421 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:37.606396    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:28:37.610854 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:37.610828    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:28:37.614036 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:37.614019    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:28:37.618473 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:37.618455    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:28:50.498078 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:50.498047    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:28:51.131976 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:51.131943    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:28:54.805610 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:54.805576    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:28:56.562593 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:56.562545    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:28:56.618342 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:56.618301    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:28:56.827590 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:56.827512    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:28:57.253261 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:28:57.253220    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:04.691256 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:04.691229    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:07.638814 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.638776    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"]
Sep 07 09:29:07.639546 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.639514    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p" podUID="6a39d117-58c6-45f4-8c7a-909e886f0027" containerName="maas-api" containerID="cri-o://92788330236cfe8201da0b261164b2763b961454f404a60fc18139d263ac0a3f" gracePeriod=30
Sep 07 09:29:07.668704 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.668674    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:07.768592 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.768542    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:07.854383 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.854354    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:07.890374 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.890322    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:29:07.916638 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.916617    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:07.979767 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.979747    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a39d117-58c6-45f4-8c7a-909e886f0027-openshift-service-ca\") pod \"6a39d117-58c6-45f4-8c7a-909e886f0027\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") "
Sep 07 09:29:07.979899 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.979821    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdhwl\" (UniqueName: \"kubernetes.io/projected/6a39d117-58c6-45f4-8c7a-909e886f0027-kube-api-access-xdhwl\") pod \"6a39d117-58c6-45f4-8c7a-909e886f0027\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") "
Sep 07 09:29:07.979899 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.979851    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a39d117-58c6-45f4-8c7a-909e886f0027-metrics-certs\") pod \"6a39d117-58c6-45f4-8c7a-909e886f0027\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") "
Sep 07 09:29:07.980036 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.979907    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a39d117-58c6-45f4-8c7a-909e886f0027-maas-api-tls\") pod \"6a39d117-58c6-45f4-8c7a-909e886f0027\" (UID: \"6a39d117-58c6-45f4-8c7a-909e886f0027\") "
Sep 07 09:29:07.980850 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.980180    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a39d117-58c6-45f4-8c7a-909e886f0027-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6a39d117-58c6-45f4-8c7a-909e886f0027" (UID: "6a39d117-58c6-45f4-8c7a-909e886f0027"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:29:07.982691 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.982655    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a39d117-58c6-45f4-8c7a-909e886f0027-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6a39d117-58c6-45f4-8c7a-909e886f0027" (UID: "6a39d117-58c6-45f4-8c7a-909e886f0027"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:29:07.983231 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.983196    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a39d117-58c6-45f4-8c7a-909e886f0027-kube-api-access-xdhwl" (OuterVolumeSpecName: "kube-api-access-xdhwl") pod "6a39d117-58c6-45f4-8c7a-909e886f0027" (UID: "6a39d117-58c6-45f4-8c7a-909e886f0027"). InnerVolumeSpecName "kube-api-access-xdhwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:29:07.983360 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:07.983256    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a39d117-58c6-45f4-8c7a-909e886f0027-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6a39d117-58c6-45f4-8c7a-909e886f0027" (UID: "6a39d117-58c6-45f4-8c7a-909e886f0027"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:29:08.081117 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:08.081094    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a39d117-58c6-45f4-8c7a-909e886f0027-openshift-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:29:08.081117 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:08.081118    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdhwl\" (UniqueName: \"kubernetes.io/projected/6a39d117-58c6-45f4-8c7a-909e886f0027-kube-api-access-xdhwl\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:29:08.081264 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:08.081127    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a39d117-58c6-45f4-8c7a-909e886f0027-metrics-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:29:08.081264 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:08.081135    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a39d117-58c6-45f4-8c7a-909e886f0027-maas-api-tls\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:29:08.231305 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:08.231276    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:08.731361 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:08.731325    2574 generic.go:356] "Generic (PLEG): container finished" podID="6a39d117-58c6-45f4-8c7a-909e886f0027" containerID="92788330236cfe8201da0b261164b2763b961454f404a60fc18139d263ac0a3f" exitCode=0
Sep 07 09:29:08.731792 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:08.731371    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"
Sep 07 09:29:08.731792 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:08.731407    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p" event={"ID":"6a39d117-58c6-45f4-8c7a-909e886f0027","Type":"ContainerDied","Data":"92788330236cfe8201da0b261164b2763b961454f404a60fc18139d263ac0a3f"}
Sep 07 09:29:08.731792 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:08.731467    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p" event={"ID":"6a39d117-58c6-45f4-8c7a-909e886f0027","Type":"ContainerDied","Data":"033bbb7107b44660f2040676f2a2da820bd85d00f9b1945d364d7b1c009d2ed1"}
Sep 07 09:29:08.731792 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:08.731489    2574 scope.go:117] "RemoveContainer" containerID="92788330236cfe8201da0b261164b2763b961454f404a60fc18139d263ac0a3f"
Sep 07 09:29:08.740959 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:08.740941    2574 scope.go:117] "RemoveContainer" containerID="92788330236cfe8201da0b261164b2763b961454f404a60fc18139d263ac0a3f"
Sep 07 09:29:08.741230 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:29:08.741210    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92788330236cfe8201da0b261164b2763b961454f404a60fc18139d263ac0a3f\": container with ID starting with 92788330236cfe8201da0b261164b2763b961454f404a60fc18139d263ac0a3f not found: ID does not exist" containerID="92788330236cfe8201da0b261164b2763b961454f404a60fc18139d263ac0a3f"
Sep 07 09:29:08.741364 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:08.741241    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92788330236cfe8201da0b261164b2763b961454f404a60fc18139d263ac0a3f"} err="failed to get container status \"92788330236cfe8201da0b261164b2763b961454f404a60fc18139d263ac0a3f\": rpc error: code = NotFound desc = could not find container \"92788330236cfe8201da0b261164b2763b961454f404a60fc18139d263ac0a3f\": container with ID starting with 92788330236cfe8201da0b261164b2763b961454f404a60fc18139d263ac0a3f not found: ID does not exist"
Sep 07 09:29:08.754461 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:08.754437    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"]
Sep 07 09:29:08.759901 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:08.759881    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-da0a32-56d6857fbf-2926p"]
Sep 07 09:29:09.542848 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:09.542811    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a39d117-58c6-45f4-8c7a-909e886f0027" path="/var/lib/kubelet/pods/6a39d117-58c6-45f4-8c7a-909e886f0027/volumes"
Sep 07 09:29:10.701887 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:10.701859    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:29:14.263438 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:14.263397    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:15.814151 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:15.814109    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:23.967922 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:23.967891    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:24.676929 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:24.676881    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:25.210446 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.210417    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"]
Sep 07 09:29:25.210883 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.210610    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk" podUID="ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa" containerName="maas-api" containerID="cri-o://0d89af4bfef86981c146aee668b2f4d9fc644b1b1f3161bfdc228988096183df" gracePeriod=30
Sep 07 09:29:25.226449 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.226418    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:25.309613 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.309585    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:25.382943 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.382914    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:25.460371 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.460348    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:29:25.467946 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.467878    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:25.528341 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.528315    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-maas-api-tls\") pod \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") "
Sep 07 09:29:25.528449 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.528380    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-metrics-certs\") pod \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") "
Sep 07 09:29:25.528449 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.528402    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mvkw\" (UniqueName: \"kubernetes.io/projected/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-kube-api-access-4mvkw\") pod \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") "
Sep 07 09:29:25.528532 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.528464    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-openshift-service-ca\") pod \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\" (UID: \"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa\") "
Sep 07 09:29:25.528905 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.528878    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa" (UID: "ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:29:25.530520 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.530501    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa" (UID: "ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:29:25.530618 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.530554    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa" (UID: "ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:29:25.530618 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.530587    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-kube-api-access-4mvkw" (OuterVolumeSpecName: "kube-api-access-4mvkw") pod "ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa" (UID: "ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa"). InnerVolumeSpecName "kube-api-access-4mvkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:29:25.630050 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.630023    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-maas-api-tls\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:29:25.630050 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.630053    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-metrics-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:29:25.630262 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.630068    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mvkw\" (UniqueName: \"kubernetes.io/projected/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-kube-api-access-4mvkw\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:29:25.630262 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.630082    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa-openshift-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:29:25.815254 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.815167    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:25.817996 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.817974    2574 generic.go:356] "Generic (PLEG): container finished" podID="ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa" containerID="0d89af4bfef86981c146aee668b2f4d9fc644b1b1f3161bfdc228988096183df" exitCode=0
Sep 07 09:29:25.818120 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.818026    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"
Sep 07 09:29:25.818120 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.818052    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk" event={"ID":"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa","Type":"ContainerDied","Data":"0d89af4bfef86981c146aee668b2f4d9fc644b1b1f3161bfdc228988096183df"}
Sep 07 09:29:25.818120 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.818111    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk" event={"ID":"ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa","Type":"ContainerDied","Data":"1ec595c607f4d91f6c262b968c117118c063c8ed9eac4897b829efb00a2b91b6"}
Sep 07 09:29:25.818223 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.818135    2574 scope.go:117] "RemoveContainer" containerID="0d89af4bfef86981c146aee668b2f4d9fc644b1b1f3161bfdc228988096183df"
Sep 07 09:29:25.826530 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.826515    2574 scope.go:117] "RemoveContainer" containerID="0d89af4bfef86981c146aee668b2f4d9fc644b1b1f3161bfdc228988096183df"
Sep 07 09:29:25.826817 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:29:25.826799    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d89af4bfef86981c146aee668b2f4d9fc644b1b1f3161bfdc228988096183df\": container with ID starting with 0d89af4bfef86981c146aee668b2f4d9fc644b1b1f3161bfdc228988096183df not found: ID does not exist" containerID="0d89af4bfef86981c146aee668b2f4d9fc644b1b1f3161bfdc228988096183df"
Sep 07 09:29:25.826877 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.826824    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d89af4bfef86981c146aee668b2f4d9fc644b1b1f3161bfdc228988096183df"} err="failed to get container status \"0d89af4bfef86981c146aee668b2f4d9fc644b1b1f3161bfdc228988096183df\": rpc error: code = NotFound desc = could not find container \"0d89af4bfef86981c146aee668b2f4d9fc644b1b1f3161bfdc228988096183df\": container with ID starting with 0d89af4bfef86981c146aee668b2f4d9fc644b1b1f3161bfdc228988096183df not found: ID does not exist"
Sep 07 09:29:25.842856 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.842823    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"]
Sep 07 09:29:25.847044 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:25.847024    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-3dc031-f8f74644d-gr7pk"]
Sep 07 09:29:27.542539 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:27.542503    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa" path="/var/lib/kubelet/pods/ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa/volumes"
Sep 07 09:29:28.214623 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:28.214572    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:29:31.546314 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:31.546274    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:31.608944 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:31.608917    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:31.820595 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:31.820516    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:42.472149 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.472120    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"]
Sep 07 09:29:42.472552 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.472341    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj" podUID="7d06c2d4-8ee5-4325-a927-f370aa73858f" containerName="maas-api" containerID="cri-o://7bc31f9cc70f35dfdc4eaf41c979f72ca911dadcd5193c6f83fe0b9c20c1702d" gracePeriod=30
Sep 07 09:29:42.508336 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.508311    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:42.605531 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.605497    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:42.676738 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.676690    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:42.725065 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.725011    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:29:42.728581 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.728559    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:42.771590 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.771562    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77fdg\" (UniqueName: \"kubernetes.io/projected/7d06c2d4-8ee5-4325-a927-f370aa73858f-kube-api-access-77fdg\") pod \"7d06c2d4-8ee5-4325-a927-f370aa73858f\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") "
Sep 07 09:29:42.771734 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.771622    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d06c2d4-8ee5-4325-a927-f370aa73858f-metrics-certs\") pod \"7d06c2d4-8ee5-4325-a927-f370aa73858f\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") "
Sep 07 09:29:42.771734 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.771653    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d06c2d4-8ee5-4325-a927-f370aa73858f-maas-api-tls\") pod \"7d06c2d4-8ee5-4325-a927-f370aa73858f\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") "
Sep 07 09:29:42.771734 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.771695    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d06c2d4-8ee5-4325-a927-f370aa73858f-openshift-service-ca\") pod \"7d06c2d4-8ee5-4325-a927-f370aa73858f\" (UID: \"7d06c2d4-8ee5-4325-a927-f370aa73858f\") "
Sep 07 09:29:42.772124 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.772099    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d06c2d4-8ee5-4325-a927-f370aa73858f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7d06c2d4-8ee5-4325-a927-f370aa73858f" (UID: "7d06c2d4-8ee5-4325-a927-f370aa73858f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:29:42.773777 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.773750    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d06c2d4-8ee5-4325-a927-f370aa73858f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7d06c2d4-8ee5-4325-a927-f370aa73858f" (UID: "7d06c2d4-8ee5-4325-a927-f370aa73858f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:29:42.773777 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.773756    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d06c2d4-8ee5-4325-a927-f370aa73858f-kube-api-access-77fdg" (OuterVolumeSpecName: "kube-api-access-77fdg") pod "7d06c2d4-8ee5-4325-a927-f370aa73858f" (UID: "7d06c2d4-8ee5-4325-a927-f370aa73858f"). InnerVolumeSpecName "kube-api-access-77fdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:29:42.773777 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.773770    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d06c2d4-8ee5-4325-a927-f370aa73858f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7d06c2d4-8ee5-4325-a927-f370aa73858f" (UID: "7d06c2d4-8ee5-4325-a927-f370aa73858f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:29:42.872955 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.872930    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d06c2d4-8ee5-4325-a927-f370aa73858f-maas-api-tls\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:29:42.872955 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.872952    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d06c2d4-8ee5-4325-a927-f370aa73858f-openshift-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:29:42.873096 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.872961    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77fdg\" (UniqueName: \"kubernetes.io/projected/7d06c2d4-8ee5-4325-a927-f370aa73858f-kube-api-access-77fdg\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:29:42.873096 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.872971    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d06c2d4-8ee5-4325-a927-f370aa73858f-metrics-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:29:42.907661 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.907634    2574 generic.go:356] "Generic (PLEG): container finished" podID="7d06c2d4-8ee5-4325-a927-f370aa73858f" containerID="7bc31f9cc70f35dfdc4eaf41c979f72ca911dadcd5193c6f83fe0b9c20c1702d" exitCode=0
Sep 07 09:29:42.907774 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.907668    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj" event={"ID":"7d06c2d4-8ee5-4325-a927-f370aa73858f","Type":"ContainerDied","Data":"7bc31f9cc70f35dfdc4eaf41c979f72ca911dadcd5193c6f83fe0b9c20c1702d"}
Sep 07 09:29:42.907774 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.907689    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"
Sep 07 09:29:42.907885 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.907772    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj" event={"ID":"7d06c2d4-8ee5-4325-a927-f370aa73858f","Type":"ContainerDied","Data":"fd4f50d608446ebbebea87877c33014c0e8a27f5c1bf8cdc6e0e5b7e6ebaa510"}
Sep 07 09:29:42.907885 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.907797    2574 scope.go:117] "RemoveContainer" containerID="7bc31f9cc70f35dfdc4eaf41c979f72ca911dadcd5193c6f83fe0b9c20c1702d"
Sep 07 09:29:42.916854 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.916834    2574 scope.go:117] "RemoveContainer" containerID="7bc31f9cc70f35dfdc4eaf41c979f72ca911dadcd5193c6f83fe0b9c20c1702d"
Sep 07 09:29:42.917102 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:29:42.917085    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bc31f9cc70f35dfdc4eaf41c979f72ca911dadcd5193c6f83fe0b9c20c1702d\": container with ID starting with 7bc31f9cc70f35dfdc4eaf41c979f72ca911dadcd5193c6f83fe0b9c20c1702d not found: ID does not exist" containerID="7bc31f9cc70f35dfdc4eaf41c979f72ca911dadcd5193c6f83fe0b9c20c1702d"
Sep 07 09:29:42.917164 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.917107    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bc31f9cc70f35dfdc4eaf41c979f72ca911dadcd5193c6f83fe0b9c20c1702d"} err="failed to get container status \"7bc31f9cc70f35dfdc4eaf41c979f72ca911dadcd5193c6f83fe0b9c20c1702d\": rpc error: code = NotFound desc = could not find container \"7bc31f9cc70f35dfdc4eaf41c979f72ca911dadcd5193c6f83fe0b9c20c1702d\": container with ID starting with 7bc31f9cc70f35dfdc4eaf41c979f72ca911dadcd5193c6f83fe0b9c20c1702d not found: ID does not exist"
Sep 07 09:29:42.927116 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.927096    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"]
Sep 07 09:29:42.930557 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:42.930537    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-34de95-847d97f959-qkjpj"]
Sep 07 09:29:43.072376 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:43.072309    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:43.542444 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:43.542410    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d06c2d4-8ee5-4325-a927-f370aa73858f" path="/var/lib/kubelet/pods/7d06c2d4-8ee5-4325-a927-f370aa73858f/volumes"
Sep 07 09:29:44.851909 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:44.851877    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:44.906430 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:44.906403    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:45.498525 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:45.498480    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:29:48.978022 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:48.977993    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:49.049576 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:49.049539    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:49.224740 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:49.224695    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:29:52.793266 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:29:52.793225    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:31:13.061408 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:31:13.061373    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:31:13.989339 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:31:13.989306    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:31:14.450695 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:31:14.450660    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:31:16.004985 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:31:16.004946    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:31:40.198586 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:31:40.198546    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:31:47.899027 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:31:47.898993    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:31:57.299514 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:31:57.299478    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:32:04.693952 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:04.693915    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:32:15.591520 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:15.591484    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:32:21.219330 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:21.219298    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"]
Sep 07 09:32:21.220059 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:21.219493    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb" podUID="6c18adc6-4851-4992-b4ab-12c3ad378dfe" containerName="manager" containerID="cri-o://9ceea972a9f8e27b80f024de8ea431b6b22f8c02a71d8f7030c3bdc7d8848640" gracePeriod=10
Sep 07 09:32:22.697764 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:22.697698    2574 generic.go:356] "Generic (PLEG): container finished" podID="6c18adc6-4851-4992-b4ab-12c3ad378dfe" containerID="9ceea972a9f8e27b80f024de8ea431b6b22f8c02a71d8f7030c3bdc7d8848640" exitCode=0
Sep 07 09:32:22.698102 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:22.697771    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb" event={"ID":"6c18adc6-4851-4992-b4ab-12c3ad378dfe","Type":"ContainerDied","Data":"9ceea972a9f8e27b80f024de8ea431b6b22f8c02a71d8f7030c3bdc7d8848640"}
Sep 07 09:32:23.279078 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:23.279056    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"
Sep 07 09:32:23.350465 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:23.350434    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c18adc6-4851-4992-b4ab-12c3ad378dfe-extensions-socket-volume\") pod \"6c18adc6-4851-4992-b4ab-12c3ad378dfe\" (UID: \"6c18adc6-4851-4992-b4ab-12c3ad378dfe\") "
Sep 07 09:32:23.350636 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:23.350559    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5xrv\" (UniqueName: \"kubernetes.io/projected/6c18adc6-4851-4992-b4ab-12c3ad378dfe-kube-api-access-v5xrv\") pod \"6c18adc6-4851-4992-b4ab-12c3ad378dfe\" (UID: \"6c18adc6-4851-4992-b4ab-12c3ad378dfe\") "
Sep 07 09:32:23.350857 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:23.350833    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c18adc6-4851-4992-b4ab-12c3ad378dfe-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6c18adc6-4851-4992-b4ab-12c3ad378dfe" (UID: "6c18adc6-4851-4992-b4ab-12c3ad378dfe"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:32:23.352870 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:23.352837    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c18adc6-4851-4992-b4ab-12c3ad378dfe-kube-api-access-v5xrv" (OuterVolumeSpecName: "kube-api-access-v5xrv") pod "6c18adc6-4851-4992-b4ab-12c3ad378dfe" (UID: "6c18adc6-4851-4992-b4ab-12c3ad378dfe"). InnerVolumeSpecName "kube-api-access-v5xrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:32:23.451529 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:23.451498    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5xrv\" (UniqueName: \"kubernetes.io/projected/6c18adc6-4851-4992-b4ab-12c3ad378dfe-kube-api-access-v5xrv\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:32:23.451529 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:23.451526    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c18adc6-4851-4992-b4ab-12c3ad378dfe-extensions-socket-volume\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:32:23.704068 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:23.704010    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"
Sep 07 09:32:23.704068 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:23.704054    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb" event={"ID":"6c18adc6-4851-4992-b4ab-12c3ad378dfe","Type":"ContainerDied","Data":"c6cfcf7543aabbb34f3cfb233472e0638a91cc48da85bbad7ec854b764903717"}
Sep 07 09:32:23.704429 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:23.704102    2574 scope.go:117] "RemoveContainer" containerID="9ceea972a9f8e27b80f024de8ea431b6b22f8c02a71d8f7030c3bdc7d8848640"
Sep 07 09:32:23.719742 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:23.719701    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"]
Sep 07 09:32:23.724844 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:23.724825    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8rrhb"]
Sep 07 09:32:25.542535 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:25.542501    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c18adc6-4851-4992-b4ab-12c3ad378dfe" path="/var/lib/kubelet/pods/6c18adc6-4851-4992-b4ab-12c3ad378dfe/volumes"
Sep 07 09:32:29.856339 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:32:29.856293    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:33:07.396833 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:33:07.396753    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:33:07.446296 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:33:07.446267    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:33:37.641187 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:33:37.641155    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:33:37.647243 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:33:37.647221    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:33:37.649386 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:33:37.649362    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:33:37.654853 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:33:37.654834    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:33:41.793512 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:33:41.793478    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:33:49.900086 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:33:49.900050    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:33:55.800846 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:33:55.800813    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:34:13.688275 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688243    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"]
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688625    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d06c2d4-8ee5-4325-a927-f370aa73858f" containerName="maas-api"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688638    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d06c2d4-8ee5-4325-a927-f370aa73858f" containerName="maas-api"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688649    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa" containerName="maas-api"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688656    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa" containerName="maas-api"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688664    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bd75921-f1b0-41ba-9780-8ed6850919f8" containerName="main"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688670    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bd75921-f1b0-41ba-9780-8ed6850919f8" containerName="main"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688678    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c18adc6-4851-4992-b4ab-12c3ad378dfe" containerName="manager"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688683    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c18adc6-4851-4992-b4ab-12c3ad378dfe" containerName="manager"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688696    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f7b84ef-f29a-456c-83ff-631ab822b2b6" containerName="maas-api"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688701    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f7b84ef-f29a-456c-83ff-631ab822b2b6" containerName="maas-api"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688732    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a39d117-58c6-45f4-8c7a-909e886f0027" containerName="maas-api"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688738    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a39d117-58c6-45f4-8c7a-909e886f0027" containerName="maas-api"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688797    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea5f9dc1-8fa0-4de1-a15c-5ca5bd848baa" containerName="maas-api"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688809    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bd75921-f1b0-41ba-9780-8ed6850919f8" containerName="main"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688817    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d06c2d4-8ee5-4325-a927-f370aa73858f" containerName="maas-api"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688822    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a39d117-58c6-45f4-8c7a-909e886f0027" containerName="maas-api"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688829    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f7b84ef-f29a-456c-83ff-631ab822b2b6" containerName="maas-api"
Sep 07 09:34:13.689798 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.688834    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c18adc6-4851-4992-b4ab-12c3ad378dfe" containerName="manager"
Sep 07 09:34:13.690939 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.690922    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.692876 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.692854    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-ab5c71d0-kserve-self-signed-certs\""
Sep 07 09:34:13.698557 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.698535    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"]
Sep 07 09:34:13.720541 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.720518    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:34:13.780892 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.780868    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-dshm\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.780981 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.780903    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-model-cache\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.780981 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.780926    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-tmp-dir\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.781052 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.780997    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee611deb-910d-46f7-860f-807524e2fb1b-tls-certs\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.781052 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.781025    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8xrg\" (UniqueName: \"kubernetes.io/projected/ee611deb-910d-46f7-860f-807524e2fb1b-kube-api-access-d8xrg\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.781127 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.781094    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-home\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.882317 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.882283    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-model-cache\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.882317 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.882319    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-tmp-dir\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.882523 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.882339    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee611deb-910d-46f7-860f-807524e2fb1b-tls-certs\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.882523 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.882359    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8xrg\" (UniqueName: \"kubernetes.io/projected/ee611deb-910d-46f7-860f-807524e2fb1b-kube-api-access-d8xrg\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.882523 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.882405    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-home\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.882523 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.882442    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-dshm\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.882764 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.882738    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-tmp-dir\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.882864 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.882795    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-model-cache\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.882934 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.882865    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-home\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.884914 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.884892    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-dshm\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.885064 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.885049    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee611deb-910d-46f7-860f-807524e2fb1b-tls-certs\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:13.889925 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:13.889901    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8xrg\" (UniqueName: \"kubernetes.io/projected/ee611deb-910d-46f7-860f-807524e2fb1b-kube-api-access-d8xrg\") pod \"e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") " pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:14.001040 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:14.000951    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:14.272636 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:14.272611    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"]
Sep 07 09:34:14.274692 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:34:14.274671    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee611deb_910d_46f7_860f_807524e2fb1b.slice/crio-19810f6c011baa12d040d0a8fd015c2142c2cfa77dc25aed897740c348552fdd WatchSource:0}: Error finding container 19810f6c011baa12d040d0a8fd015c2142c2cfa77dc25aed897740c348552fdd: Status 404 returned error can't find the container with id 19810f6c011baa12d040d0a8fd015c2142c2cfa77dc25aed897740c348552fdd
Sep 07 09:34:15.227682 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:15.227641    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt" event={"ID":"ee611deb-910d-46f7-860f-807524e2fb1b","Type":"ContainerStarted","Data":"417e4f32f588b0011f1a4b05088215daaf3bc5c5a1ecc55eb384ae1a332005db"}
Sep 07 09:34:15.228190 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:15.227695    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt" event={"ID":"ee611deb-910d-46f7-860f-807524e2fb1b","Type":"ContainerStarted","Data":"19810f6c011baa12d040d0a8fd015c2142c2cfa77dc25aed897740c348552fdd"}
Sep 07 09:34:15.243068 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:15.243022    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt" podStartSLOduration=2.243007159 podStartE2EDuration="2.243007159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:34:15.242505795 +0000 UTC m=+2138.206422017" watchObservedRunningTime="2026-09-07 09:34:15.243007159 +0000 UTC m=+2138.206923380"
Sep 07 09:34:16.398601 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:16.398568    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:34:20.783049 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.783014    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"]
Sep 07 09:34:20.785588 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.785571    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.787612 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.787591    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-ab5c71d0-kserve-self-signed-certs\""
Sep 07 09:34:20.793574 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.793424    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"]
Sep 07 09:34:20.805189 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.805163    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:34:20.842167 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.842144    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-tmp-dir\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.842271 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.842186    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25230f77-0881-4b24-8768-fd618ce8dd9e-tls-certs\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.842271 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.842208    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-home\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.842271 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.842226    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-dshm\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.842271 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.842247    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-model-cache\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.842439 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.842296    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpks4\" (UniqueName: \"kubernetes.io/projected/25230f77-0881-4b24-8768-fd618ce8dd9e-kube-api-access-rpks4\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.942932 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.942903    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-tmp-dir\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.943059 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.942950    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25230f77-0881-4b24-8768-fd618ce8dd9e-tls-certs\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.943059 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.942968    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-home\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.943059 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.942984    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-dshm\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.943059 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.943001    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-model-cache\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.943059 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.943034    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpks4\" (UniqueName: \"kubernetes.io/projected/25230f77-0881-4b24-8768-fd618ce8dd9e-kube-api-access-rpks4\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.943361 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.943336    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-tmp-dir\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.943440 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.943367    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-home\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.943495 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.943439    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-model-cache\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.945416 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.945394    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-dshm\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.945637 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.945620    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25230f77-0881-4b24-8768-fd618ce8dd9e-tls-certs\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:20.950663 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:20.950645    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpks4\" (UniqueName: \"kubernetes.io/projected/25230f77-0881-4b24-8768-fd618ce8dd9e-kube-api-access-rpks4\") pod \"e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") " pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:21.096076 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:21.095988    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:21.363181 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:21.363152    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"]
Sep 07 09:34:21.364906 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:34:21.364878    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25230f77_0881_4b24_8768_fd618ce8dd9e.slice/crio-1c99a5a9ff75cd04fd053394e2ae3de807f43c6b4b1606b26a5867ae184d3d2e WatchSource:0}: Error finding container 1c99a5a9ff75cd04fd053394e2ae3de807f43c6b4b1606b26a5867ae184d3d2e: Status 404 returned error can't find the container with id 1c99a5a9ff75cd04fd053394e2ae3de807f43c6b4b1606b26a5867ae184d3d2e
Sep 07 09:34:22.262338 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:22.262293    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj" event={"ID":"25230f77-0881-4b24-8768-fd618ce8dd9e","Type":"ContainerStarted","Data":"91450a5730644fdfdef83bdc29bcedd1fd002956249e3ac8829a3970f215deb1"}
Sep 07 09:34:22.262338 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:22.262344    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj" event={"ID":"25230f77-0881-4b24-8768-fd618ce8dd9e","Type":"ContainerStarted","Data":"1c99a5a9ff75cd04fd053394e2ae3de807f43c6b4b1606b26a5867ae184d3d2e"}
Sep 07 09:34:22.278080 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:22.278032    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj" podStartSLOduration=2.278017568 podStartE2EDuration="2.278017568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:34:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:34:22.275839157 +0000 UTC m=+2145.239755379" watchObservedRunningTime="2026-09-07 09:34:22.278017568 +0000 UTC m=+2145.241933798"
Sep 07 09:34:23.896283 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:23.896249    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:34:24.001810 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:24.001777    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:24.001969 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:24.001832    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:24.014363 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:24.014341    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:24.284947 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:24.284923    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:34:31.096214 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:31.096118    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:31.096214 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:31.096177    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:31.108864 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:31.108840    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:31.318137 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:31.318108    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:34:50.802983 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:50.802949    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"]
Sep 07 09:34:50.826414 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:50.826389    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:34:52.585966 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:52.585932    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"]
Sep 07 09:34:52.605472 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:52.605446    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:34:52.901398 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:34:52.901325    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:35:01.400903 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:01.400871    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:35:05.824098 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:05.824061    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt" podUID="ee611deb-910d-46f7-860f-807524e2fb1b" containerName="main" containerID="cri-o://417e4f32f588b0011f1a4b05088215daaf3bc5c5a1ecc55eb384ae1a332005db" gracePeriod=45
Sep 07 09:35:06.071562 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.071539    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:35:06.125114 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.125057    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-dshm\") pod \"ee611deb-910d-46f7-860f-807524e2fb1b\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") "
Sep 07 09:35:06.125114 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.125090    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-model-cache\") pod \"ee611deb-910d-46f7-860f-807524e2fb1b\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") "
Sep 07 09:35:06.125253 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.125135    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-home\") pod \"ee611deb-910d-46f7-860f-807524e2fb1b\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") "
Sep 07 09:35:06.125253 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.125169    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee611deb-910d-46f7-860f-807524e2fb1b-tls-certs\") pod \"ee611deb-910d-46f7-860f-807524e2fb1b\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") "
Sep 07 09:35:06.125253 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.125185    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8xrg\" (UniqueName: \"kubernetes.io/projected/ee611deb-910d-46f7-860f-807524e2fb1b-kube-api-access-d8xrg\") pod \"ee611deb-910d-46f7-860f-807524e2fb1b\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") "
Sep 07 09:35:06.125253 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.125231    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-tmp-dir\") pod \"ee611deb-910d-46f7-860f-807524e2fb1b\" (UID: \"ee611deb-910d-46f7-860f-807524e2fb1b\") "
Sep 07 09:35:06.125451 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.125332    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-model-cache" (OuterVolumeSpecName: "model-cache") pod "ee611deb-910d-46f7-860f-807524e2fb1b" (UID: "ee611deb-910d-46f7-860f-807524e2fb1b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:35:06.125451 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.125400    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-home" (OuterVolumeSpecName: "home") pod "ee611deb-910d-46f7-860f-807524e2fb1b" (UID: "ee611deb-910d-46f7-860f-807524e2fb1b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:35:06.125579 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.125499    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:06.125579 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.125512    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:06.125804 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.125763    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ee611deb-910d-46f7-860f-807524e2fb1b" (UID: "ee611deb-910d-46f7-860f-807524e2fb1b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:35:06.127265 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.127247    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-dshm" (OuterVolumeSpecName: "dshm") pod "ee611deb-910d-46f7-860f-807524e2fb1b" (UID: "ee611deb-910d-46f7-860f-807524e2fb1b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:35:06.127382 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.127361    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee611deb-910d-46f7-860f-807524e2fb1b-kube-api-access-d8xrg" (OuterVolumeSpecName: "kube-api-access-d8xrg") pod "ee611deb-910d-46f7-860f-807524e2fb1b" (UID: "ee611deb-910d-46f7-860f-807524e2fb1b"). InnerVolumeSpecName "kube-api-access-d8xrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:35:06.127441 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.127406    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee611deb-910d-46f7-860f-807524e2fb1b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ee611deb-910d-46f7-860f-807524e2fb1b" (UID: "ee611deb-910d-46f7-860f-807524e2fb1b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:35:06.226817 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.226790    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee611deb-910d-46f7-860f-807524e2fb1b-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:06.226817 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.226813    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8xrg\" (UniqueName: \"kubernetes.io/projected/ee611deb-910d-46f7-860f-807524e2fb1b-kube-api-access-d8xrg\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:06.226952 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.226842    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:06.226952 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.226851    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee611deb-910d-46f7-860f-807524e2fb1b-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:06.488175 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.488137    2574 generic.go:356] "Generic (PLEG): container finished" podID="ee611deb-910d-46f7-860f-807524e2fb1b" containerID="417e4f32f588b0011f1a4b05088215daaf3bc5c5a1ecc55eb384ae1a332005db" exitCode=0
Sep 07 09:35:06.488353 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.488183    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"
Sep 07 09:35:06.488353 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.488181    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt" event={"ID":"ee611deb-910d-46f7-860f-807524e2fb1b","Type":"ContainerDied","Data":"417e4f32f588b0011f1a4b05088215daaf3bc5c5a1ecc55eb384ae1a332005db"}
Sep 07 09:35:06.488353 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.488315    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt" event={"ID":"ee611deb-910d-46f7-860f-807524e2fb1b","Type":"ContainerDied","Data":"19810f6c011baa12d040d0a8fd015c2142c2cfa77dc25aed897740c348552fdd"}
Sep 07 09:35:06.488353 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.488343    2574 scope.go:117] "RemoveContainer" containerID="417e4f32f588b0011f1a4b05088215daaf3bc5c5a1ecc55eb384ae1a332005db"
Sep 07 09:35:06.497404 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.497387    2574 scope.go:117] "RemoveContainer" containerID="417e4f32f588b0011f1a4b05088215daaf3bc5c5a1ecc55eb384ae1a332005db"
Sep 07 09:35:06.497678 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:35:06.497658    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"417e4f32f588b0011f1a4b05088215daaf3bc5c5a1ecc55eb384ae1a332005db\": container with ID starting with 417e4f32f588b0011f1a4b05088215daaf3bc5c5a1ecc55eb384ae1a332005db not found: ID does not exist" containerID="417e4f32f588b0011f1a4b05088215daaf3bc5c5a1ecc55eb384ae1a332005db"
Sep 07 09:35:06.497772 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.497685    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"417e4f32f588b0011f1a4b05088215daaf3bc5c5a1ecc55eb384ae1a332005db"} err="failed to get container status \"417e4f32f588b0011f1a4b05088215daaf3bc5c5a1ecc55eb384ae1a332005db\": rpc error: code = NotFound desc = could not find container \"417e4f32f588b0011f1a4b05088215daaf3bc5c5a1ecc55eb384ae1a332005db\": container with ID starting with 417e4f32f588b0011f1a4b05088215daaf3bc5c5a1ecc55eb384ae1a332005db not found: ID does not exist"
Sep 07 09:35:06.507458 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.507434    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"]
Sep 07 09:35:06.511186 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:06.511165    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-ab5c71d0-kserve-6dbd8db847-j2znt"]
Sep 07 09:35:07.543017 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.542982    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee611deb-910d-46f7-860f-807524e2fb1b" path="/var/lib/kubelet/pods/ee611deb-910d-46f7-860f-807524e2fb1b/volumes"
Sep 07 09:35:07.606098 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.606062    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj" podUID="25230f77-0881-4b24-8768-fd618ce8dd9e" containerName="main" containerID="cri-o://91450a5730644fdfdef83bdc29bcedd1fd002956249e3ac8829a3970f215deb1" gracePeriod=45
Sep 07 09:35:07.857910 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.857885    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:35:07.941026 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.940999    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-dshm\") pod \"25230f77-0881-4b24-8768-fd618ce8dd9e\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") "
Sep 07 09:35:07.941175 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.941035    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-home\") pod \"25230f77-0881-4b24-8768-fd618ce8dd9e\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") "
Sep 07 09:35:07.941175 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.941079    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-tmp-dir\") pod \"25230f77-0881-4b24-8768-fd618ce8dd9e\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") "
Sep 07 09:35:07.941175 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.941139    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25230f77-0881-4b24-8768-fd618ce8dd9e-tls-certs\") pod \"25230f77-0881-4b24-8768-fd618ce8dd9e\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") "
Sep 07 09:35:07.941175 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.941170    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpks4\" (UniqueName: \"kubernetes.io/projected/25230f77-0881-4b24-8768-fd618ce8dd9e-kube-api-access-rpks4\") pod \"25230f77-0881-4b24-8768-fd618ce8dd9e\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") "
Sep 07 09:35:07.941382 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.941198    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-model-cache\") pod \"25230f77-0881-4b24-8768-fd618ce8dd9e\" (UID: \"25230f77-0881-4b24-8768-fd618ce8dd9e\") "
Sep 07 09:35:07.941382 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.941342    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-home" (OuterVolumeSpecName: "home") pod "25230f77-0881-4b24-8768-fd618ce8dd9e" (UID: "25230f77-0881-4b24-8768-fd618ce8dd9e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:35:07.941382 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.941356    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "25230f77-0881-4b24-8768-fd618ce8dd9e" (UID: "25230f77-0881-4b24-8768-fd618ce8dd9e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:35:07.941529 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.941515    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-model-cache" (OuterVolumeSpecName: "model-cache") pod "25230f77-0881-4b24-8768-fd618ce8dd9e" (UID: "25230f77-0881-4b24-8768-fd618ce8dd9e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:35:07.941617 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.941575    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-home\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:07.941617 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.941591    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-tmp-dir\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:07.941617 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.941604    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-model-cache\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:07.943396 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.943373    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25230f77-0881-4b24-8768-fd618ce8dd9e-kube-api-access-rpks4" (OuterVolumeSpecName: "kube-api-access-rpks4") pod "25230f77-0881-4b24-8768-fd618ce8dd9e" (UID: "25230f77-0881-4b24-8768-fd618ce8dd9e"). InnerVolumeSpecName "kube-api-access-rpks4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:35:07.943867 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.943846    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-dshm" (OuterVolumeSpecName: "dshm") pod "25230f77-0881-4b24-8768-fd618ce8dd9e" (UID: "25230f77-0881-4b24-8768-fd618ce8dd9e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 09:35:07.943949 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:07.943848    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25230f77-0881-4b24-8768-fd618ce8dd9e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "25230f77-0881-4b24-8768-fd618ce8dd9e" (UID: "25230f77-0881-4b24-8768-fd618ce8dd9e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:35:08.042900 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:08.042872    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25230f77-0881-4b24-8768-fd618ce8dd9e-tls-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:08.042900 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:08.042897    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpks4\" (UniqueName: \"kubernetes.io/projected/25230f77-0881-4b24-8768-fd618ce8dd9e-kube-api-access-rpks4\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:08.043039 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:08.042911    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25230f77-0881-4b24-8768-fd618ce8dd9e-dshm\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:08.295339 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:08.295312    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:35:08.499646 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:08.499605    2574 generic.go:356] "Generic (PLEG): container finished" podID="25230f77-0881-4b24-8768-fd618ce8dd9e" containerID="91450a5730644fdfdef83bdc29bcedd1fd002956249e3ac8829a3970f215deb1" exitCode=0
Sep 07 09:35:08.499844 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:08.499651    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"
Sep 07 09:35:08.499844 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:08.499689    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj" event={"ID":"25230f77-0881-4b24-8768-fd618ce8dd9e","Type":"ContainerDied","Data":"91450a5730644fdfdef83bdc29bcedd1fd002956249e3ac8829a3970f215deb1"}
Sep 07 09:35:08.499844 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:08.499752    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj" event={"ID":"25230f77-0881-4b24-8768-fd618ce8dd9e","Type":"ContainerDied","Data":"1c99a5a9ff75cd04fd053394e2ae3de807f43c6b4b1606b26a5867ae184d3d2e"}
Sep 07 09:35:08.499844 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:08.499772    2574 scope.go:117] "RemoveContainer" containerID="91450a5730644fdfdef83bdc29bcedd1fd002956249e3ac8829a3970f215deb1"
Sep 07 09:35:08.508822 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:08.508806    2574 scope.go:117] "RemoveContainer" containerID="91450a5730644fdfdef83bdc29bcedd1fd002956249e3ac8829a3970f215deb1"
Sep 07 09:35:08.509052 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:35:08.509034    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91450a5730644fdfdef83bdc29bcedd1fd002956249e3ac8829a3970f215deb1\": container with ID starting with 91450a5730644fdfdef83bdc29bcedd1fd002956249e3ac8829a3970f215deb1 not found: ID does not exist" containerID="91450a5730644fdfdef83bdc29bcedd1fd002956249e3ac8829a3970f215deb1"
Sep 07 09:35:08.509094 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:08.509057    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91450a5730644fdfdef83bdc29bcedd1fd002956249e3ac8829a3970f215deb1"} err="failed to get container status \"91450a5730644fdfdef83bdc29bcedd1fd002956249e3ac8829a3970f215deb1\": rpc error: code = NotFound desc = could not find container \"91450a5730644fdfdef83bdc29bcedd1fd002956249e3ac8829a3970f215deb1\": container with ID starting with 91450a5730644fdfdef83bdc29bcedd1fd002956249e3ac8829a3970f215deb1 not found: ID does not exist"
Sep 07 09:35:08.519360 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:08.519339    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"]
Sep 07 09:35:08.523549 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:08.523529    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-ab5c71d0-kserve-5f644668ff-7rmmj"]
Sep 07 09:35:09.541420 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:09.541382    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25230f77-0881-4b24-8768-fd618ce8dd9e" path="/var/lib/kubelet/pods/25230f77-0881-4b24-8768-fd618ce8dd9e/volumes"
Sep 07 09:35:16.093059 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.093030    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"]
Sep 07 09:35:16.093402 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.093380    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee611deb-910d-46f7-860f-807524e2fb1b" containerName="main"
Sep 07 09:35:16.093402 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.093392    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee611deb-910d-46f7-860f-807524e2fb1b" containerName="main"
Sep 07 09:35:16.093402 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.093402    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25230f77-0881-4b24-8768-fd618ce8dd9e" containerName="main"
Sep 07 09:35:16.093536 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.093409    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="25230f77-0881-4b24-8768-fd618ce8dd9e" containerName="main"
Sep 07 09:35:16.093536 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.093468    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee611deb-910d-46f7-860f-807524e2fb1b" containerName="main"
Sep 07 09:35:16.093536 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.093474    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="25230f77-0881-4b24-8768-fd618ce8dd9e" containerName="main"
Sep 07 09:35:16.098075 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.098054    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:16.100181 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.100159    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 07 09:35:16.100737 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.100702    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rckvc\""
Sep 07 09:35:16.100847 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.100778    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 09:35:16.100847 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.100812    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 07 09:35:16.100847 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.100831    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 07 09:35:16.106239 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.106216    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"]
Sep 07 09:35:16.111065 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.111047    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"]
Sep 07 09:35:16.117137 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.117121    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:16.126243 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.126221    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"]
Sep 07 09:35:16.192265 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.192237    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:35:16.214036 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.214007    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203ec927-f760-4774-9db1-a01f74422f08-maas-api-tls\") pod \"maas-api-5877fc59f9-gjq5g\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:16.214176 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.214045    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551-maas-api-tls\") pod \"maas-api-7f77f9455f-4lzmg\" (UID: \"c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:16.214176 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.214064    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551-openshift-service-ca\") pod \"maas-api-7f77f9455f-4lzmg\" (UID: \"c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:16.214176 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.214082    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203ec927-f760-4774-9db1-a01f74422f08-metrics-certs\") pod \"maas-api-5877fc59f9-gjq5g\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:16.214176 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.214157    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzjtl\" (UniqueName: \"kubernetes.io/projected/c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551-kube-api-access-mzjtl\") pod \"maas-api-7f77f9455f-4lzmg\" (UID: \"c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:16.214345 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.214182    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551-metrics-certs\") pod \"maas-api-7f77f9455f-4lzmg\" (UID: \"c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:16.214345 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.214199    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msjfw\" (UniqueName: \"kubernetes.io/projected/203ec927-f760-4774-9db1-a01f74422f08-kube-api-access-msjfw\") pod \"maas-api-5877fc59f9-gjq5g\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:16.214345 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.214220    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203ec927-f760-4774-9db1-a01f74422f08-openshift-service-ca\") pod \"maas-api-5877fc59f9-gjq5g\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:16.315597 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.315566    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551-metrics-certs\") pod \"maas-api-7f77f9455f-4lzmg\" (UID: \"c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:16.315820 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.315603    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msjfw\" (UniqueName: \"kubernetes.io/projected/203ec927-f760-4774-9db1-a01f74422f08-kube-api-access-msjfw\") pod \"maas-api-5877fc59f9-gjq5g\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:16.315820 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.315636    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203ec927-f760-4774-9db1-a01f74422f08-openshift-service-ca\") pod \"maas-api-5877fc59f9-gjq5g\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:16.315820 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.315748    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203ec927-f760-4774-9db1-a01f74422f08-maas-api-tls\") pod \"maas-api-5877fc59f9-gjq5g\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:16.315820 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.315789    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551-maas-api-tls\") pod \"maas-api-7f77f9455f-4lzmg\" (UID: \"c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:16.315820 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.315815    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551-openshift-service-ca\") pod \"maas-api-7f77f9455f-4lzmg\" (UID: \"c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:16.316098 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.315837    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203ec927-f760-4774-9db1-a01f74422f08-metrics-certs\") pod \"maas-api-5877fc59f9-gjq5g\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:16.316098 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.315877    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzjtl\" (UniqueName: \"kubernetes.io/projected/c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551-kube-api-access-mzjtl\") pod \"maas-api-7f77f9455f-4lzmg\" (UID: \"c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:16.316477 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.316453    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203ec927-f760-4774-9db1-a01f74422f08-openshift-service-ca\") pod \"maas-api-5877fc59f9-gjq5g\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:16.316639 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.316572    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551-openshift-service-ca\") pod \"maas-api-7f77f9455f-4lzmg\" (UID: \"c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:16.318898 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.318869    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551-maas-api-tls\") pod \"maas-api-7f77f9455f-4lzmg\" (UID: \"c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:16.318980 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.318905    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203ec927-f760-4774-9db1-a01f74422f08-metrics-certs\") pod \"maas-api-5877fc59f9-gjq5g\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:16.319061 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.319042    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203ec927-f760-4774-9db1-a01f74422f08-maas-api-tls\") pod \"maas-api-5877fc59f9-gjq5g\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:16.319102 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.319043    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551-metrics-certs\") pod \"maas-api-7f77f9455f-4lzmg\" (UID: \"c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:16.323108 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.323089    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msjfw\" (UniqueName: \"kubernetes.io/projected/203ec927-f760-4774-9db1-a01f74422f08-kube-api-access-msjfw\") pod \"maas-api-5877fc59f9-gjq5g\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") " pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:16.323424 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.323405    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzjtl\" (UniqueName: \"kubernetes.io/projected/c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551-kube-api-access-mzjtl\") pod \"maas-api-7f77f9455f-4lzmg\" (UID: \"c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551\") " pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:16.410645 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.410618    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:16.428507 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.428481    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:16.684390 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.684229    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"]
Sep 07 09:35:16.687168 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:35:16.687143    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod203ec927_f760_4774_9db1_a01f74422f08.slice/crio-2e70e03f97b546d4fedcf7bc6a2e2f8c417a984e0a38b3ee0c43a1c352b55cc0 WatchSource:0}: Error finding container 2e70e03f97b546d4fedcf7bc6a2e2f8c417a984e0a38b3ee0c43a1c352b55cc0: Status 404 returned error can't find the container with id 2e70e03f97b546d4fedcf7bc6a2e2f8c417a984e0a38b3ee0c43a1c352b55cc0
Sep 07 09:35:16.688385 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.688368    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 09:35:16.709037 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:16.709018    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"]
Sep 07 09:35:16.711256 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:35:16.711233    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e9eb57_2d1e_4dd3_9726_2a9ae6f55551.slice/crio-5716c601c3285e6cf4a4383c5ddcfccf42212332a21b4fdb1946e3ec48c0adb9 WatchSource:0}: Error finding container 5716c601c3285e6cf4a4383c5ddcfccf42212332a21b4fdb1946e3ec48c0adb9: Status 404 returned error can't find the container with id 5716c601c3285e6cf4a4383c5ddcfccf42212332a21b4fdb1946e3ec48c0adb9
Sep 07 09:35:17.545009 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:17.544912    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg" event={"ID":"c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551","Type":"ContainerStarted","Data":"fb112c9dff215d50343d1a914eafa8e57565903843531dba264b21c2924ca6d9"}
Sep 07 09:35:17.545009 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:17.544974    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:17.545009 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:17.545000    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg" event={"ID":"c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551","Type":"ContainerStarted","Data":"5716c601c3285e6cf4a4383c5ddcfccf42212332a21b4fdb1946e3ec48c0adb9"}
Sep 07 09:35:17.546367 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:17.546342    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g" event={"ID":"203ec927-f760-4774-9db1-a01f74422f08","Type":"ContainerStarted","Data":"80312ff114bd5d12afb575c3253565caa0a1caa5aa8ba05966ba9c5b8b802c5d"}
Sep 07 09:35:17.546469 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:17.546385    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g" event={"ID":"203ec927-f760-4774-9db1-a01f74422f08","Type":"ContainerStarted","Data":"2e70e03f97b546d4fedcf7bc6a2e2f8c417a984e0a38b3ee0c43a1c352b55cc0"}
Sep 07 09:35:17.546469 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:17.546418    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:17.562997 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:17.562952    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg" podStartSLOduration=1.103574786 podStartE2EDuration="1.562938996s" totalImagesPullingTime="459.36421ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:35:17.561069833 +0000 UTC m=+2200.524986066" watchObservedRunningTime="2026-09-07 09:35:17.562938996 +0000 UTC m=+2200.526855217"
Sep 07 09:35:17.579244 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:17.579200    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g" podStartSLOduration=1.107181659 podStartE2EDuration="1.579187075s" totalImagesPullingTime="472.005416ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 09:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:35:17.576921611 +0000 UTC m=+2200.540837852" watchObservedRunningTime="2026-09-07 09:35:17.579187075 +0000 UTC m=+2200.543103296"
Sep 07 09:35:23.556984 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:23.556955    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f77f9455f-4lzmg"
Sep 07 09:35:23.557563 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:23.557379    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:23.616275 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:23.616246    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"]
Sep 07 09:35:23.616436 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:23.616416    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g" podUID="203ec927-f760-4774-9db1-a01f74422f08" containerName="maas-api" containerID="cri-o://80312ff114bd5d12afb575c3253565caa0a1caa5aa8ba05966ba9c5b8b802c5d" gracePeriod=30
Sep 07 09:35:23.691670 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:23.691640    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:35:23.867523 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:23.867501    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:23.977931 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:23.977905    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203ec927-f760-4774-9db1-a01f74422f08-openshift-service-ca\") pod \"203ec927-f760-4774-9db1-a01f74422f08\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") "
Sep 07 09:35:23.978076 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:23.977947    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msjfw\" (UniqueName: \"kubernetes.io/projected/203ec927-f760-4774-9db1-a01f74422f08-kube-api-access-msjfw\") pod \"203ec927-f760-4774-9db1-a01f74422f08\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") "
Sep 07 09:35:23.978076 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:23.977967    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203ec927-f760-4774-9db1-a01f74422f08-maas-api-tls\") pod \"203ec927-f760-4774-9db1-a01f74422f08\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") "
Sep 07 09:35:23.978076 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:23.978049    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203ec927-f760-4774-9db1-a01f74422f08-metrics-certs\") pod \"203ec927-f760-4774-9db1-a01f74422f08\" (UID: \"203ec927-f760-4774-9db1-a01f74422f08\") "
Sep 07 09:35:23.978362 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:23.978334    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/203ec927-f760-4774-9db1-a01f74422f08-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "203ec927-f760-4774-9db1-a01f74422f08" (UID: "203ec927-f760-4774-9db1-a01f74422f08"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 09:35:23.980239 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:23.980218    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/203ec927-f760-4774-9db1-a01f74422f08-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "203ec927-f760-4774-9db1-a01f74422f08" (UID: "203ec927-f760-4774-9db1-a01f74422f08"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:35:23.980340 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:23.980243    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/203ec927-f760-4774-9db1-a01f74422f08-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "203ec927-f760-4774-9db1-a01f74422f08" (UID: "203ec927-f760-4774-9db1-a01f74422f08"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 09:35:23.980340 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:23.980308    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/203ec927-f760-4774-9db1-a01f74422f08-kube-api-access-msjfw" (OuterVolumeSpecName: "kube-api-access-msjfw") pod "203ec927-f760-4774-9db1-a01f74422f08" (UID: "203ec927-f760-4774-9db1-a01f74422f08"). InnerVolumeSpecName "kube-api-access-msjfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 09:35:24.079415 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:24.079392    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msjfw\" (UniqueName: \"kubernetes.io/projected/203ec927-f760-4774-9db1-a01f74422f08-kube-api-access-msjfw\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:24.079415 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:24.079414    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/203ec927-f760-4774-9db1-a01f74422f08-maas-api-tls\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:24.079549 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:24.079425    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/203ec927-f760-4774-9db1-a01f74422f08-metrics-certs\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:24.079549 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:24.079435    2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/203ec927-f760-4774-9db1-a01f74422f08-openshift-service-ca\") on node \"ip-10-0-141-81.ec2.internal\" DevicePath \"\""
Sep 07 09:35:24.586032 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:24.585996    2574 generic.go:356] "Generic (PLEG): container finished" podID="203ec927-f760-4774-9db1-a01f74422f08" containerID="80312ff114bd5d12afb575c3253565caa0a1caa5aa8ba05966ba9c5b8b802c5d" exitCode=0
Sep 07 09:35:24.586478 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:24.586046    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"
Sep 07 09:35:24.586478 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:24.586082    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g" event={"ID":"203ec927-f760-4774-9db1-a01f74422f08","Type":"ContainerDied","Data":"80312ff114bd5d12afb575c3253565caa0a1caa5aa8ba05966ba9c5b8b802c5d"}
Sep 07 09:35:24.586478 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:24.586146    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g" event={"ID":"203ec927-f760-4774-9db1-a01f74422f08","Type":"ContainerDied","Data":"2e70e03f97b546d4fedcf7bc6a2e2f8c417a984e0a38b3ee0c43a1c352b55cc0"}
Sep 07 09:35:24.586478 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:24.586180    2574 scope.go:117] "RemoveContainer" containerID="80312ff114bd5d12afb575c3253565caa0a1caa5aa8ba05966ba9c5b8b802c5d"
Sep 07 09:35:24.595670 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:24.595651    2574 scope.go:117] "RemoveContainer" containerID="80312ff114bd5d12afb575c3253565caa0a1caa5aa8ba05966ba9c5b8b802c5d"
Sep 07 09:35:24.595972 ip-10-0-141-81 kubenswrapper[2574]: E0907 09:35:24.595954    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80312ff114bd5d12afb575c3253565caa0a1caa5aa8ba05966ba9c5b8b802c5d\": container with ID starting with 80312ff114bd5d12afb575c3253565caa0a1caa5aa8ba05966ba9c5b8b802c5d not found: ID does not exist" containerID="80312ff114bd5d12afb575c3253565caa0a1caa5aa8ba05966ba9c5b8b802c5d"
Sep 07 09:35:24.596031 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:24.595983    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80312ff114bd5d12afb575c3253565caa0a1caa5aa8ba05966ba9c5b8b802c5d"} err="failed to get container status \"80312ff114bd5d12afb575c3253565caa0a1caa5aa8ba05966ba9c5b8b802c5d\": rpc error: code = NotFound desc = could not find container \"80312ff114bd5d12afb575c3253565caa0a1caa5aa8ba05966ba9c5b8b802c5d\": container with ID starting with 80312ff114bd5d12afb575c3253565caa0a1caa5aa8ba05966ba9c5b8b802c5d not found: ID does not exist"
Sep 07 09:35:24.605821 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:24.605802    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"]
Sep 07 09:35:24.610087 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:24.610067    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5877fc59f9-gjq5g"]
Sep 07 09:35:25.542429 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:25.542390    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="203ec927-f760-4774-9db1-a01f74422f08" path="/var/lib/kubelet/pods/203ec927-f760-4774-9db1-a01f74422f08/volumes"
Sep 07 09:35:31.995656 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:31.995619    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:35:39.099512 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:39.099477    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:35:46.705904 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:35:46.705859    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-lwrsj"]
Sep 07 09:36:00.287912 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:00.287836    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-rw26k_16652170-c20c-4344-9e2c-55d0f861edc2/manager/0.log"
Sep 07 09:36:01.094123 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:01.094097    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c59b5944-z2bls_c077dde0-13cf-4351-a65a-14cb61f02fec/copy-manifests/0.log"
Sep 07 09:36:01.106234 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:01.106207    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c59b5944-z2bls_c077dde0-13cf-4351-a65a-14cb61f02fec/manager/0.log"
Sep 07 09:36:01.467071 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:01.467033    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7f77f9455f-4lzmg_c5e9eb57-2d1e-4dd3-9726-2a9ae6f55551/maas-api/0.log"
Sep 07 09:36:02.545596 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:02.545563    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-qdsqv_1250b781-ccde-4c52-89e6-fa72b8349f33/manager/0.log"
Sep 07 09:36:02.652631 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:02.652589    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-dkdh8_e831e301-5424-4ef1-aa22-145cce2b0656/manager/0.log"
Sep 07 09:36:02.764983 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:02.764960    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-r9tlz_aecaad15-d260-416a-ba26-c59365d9cd5d/kuadrant-console-plugin/0.log"
Sep 07 09:36:02.988114 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:02.988083    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-lwrsj_1d50742b-f01e-4c3e-9743-446f2e216402/limitador/0.log"
Sep 07 09:36:03.107203 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:03.107170    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-fbsf7_908a409f-da1f-4089-88b0-8c9651eb97a9/manager/0.log"
Sep 07 09:36:03.763115 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:03.763091    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b86566b9f-kmwb2_9ec5585a-b1fb-4a96-b88a-22dcbaa6baad/kube-auth-proxy/0.log"
Sep 07 09:36:04.240856 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:04.240828    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54ff77667d-jgqz5_9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b/router/0.log"
Sep 07 09:36:04.579935 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:04.579845    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-9mx44_f6c6b323-2416-44f8-bd0d-c31db8070ef5/main/0.log"
Sep 07 09:36:04.791990 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:04.791956    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-tvjq4_c88817c0-4c0e-4acd-96e4-fa3247282e41/main/0.log"
Sep 07 09:36:05.012266 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:05.012239    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8wvzb_89aeaf5e-978b-4fea-b38c-d75709fdce2a/main/0.log"
Sep 07 09:36:05.225358 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:05.225335    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-2jjb5_dfb234f0-e006-4d23-bda0-5cbed8f93617/main/0.log"
Sep 07 09:36:10.180198 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:10.180134    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:36:10.279875 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:10.279849    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log" limit=1024
Sep 07 09:36:13.071627 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:13.071600    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-z54bd_e3bc99a4-33c4-4a42-a55c-ef8de46f9502/global-pull-secret-syncer/0.log"
Sep 07 09:36:13.110301 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:13.110276    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-czgxz_78b45f22-f23b-46a7-b3f6-21a960bf365e/konnectivity-agent/0.log"
Sep 07 09:36:13.216095 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:13.216066    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-81.ec2.internal_9132b1637729b3158d4294075bfe2281/haproxy/0.log"
Sep 07 09:36:17.621202 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:17.621167    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-qdsqv_1250b781-ccde-4c52-89e6-fa72b8349f33/manager/0.log"
Sep 07 09:36:17.648862 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:17.648838    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-dkdh8_e831e301-5424-4ef1-aa22-145cce2b0656/manager/0.log"
Sep 07 09:36:17.680956 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:17.680928    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-r9tlz_aecaad15-d260-416a-ba26-c59365d9cd5d/kuadrant-console-plugin/0.log"
Sep 07 09:36:17.758769 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:17.758747    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-lwrsj_1d50742b-f01e-4c3e-9743-446f2e216402/limitador/0.log"
Sep 07 09:36:17.822310 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:17.822290    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-fbsf7_908a409f-da1f-4089-88b0-8c9651eb97a9/manager/0.log"
Sep 07 09:36:19.481186 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:19.481152    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-98mdq_f4d4c13f-bc7f-4f46-b32b-906e6136122b/cluster-monitoring-operator/0.log"
Sep 07 09:36:19.810452 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:19.810384    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nnb6c_b40eeca6-4402-403d-9029-79ddd8d0e073/node-exporter/0.log"
Sep 07 09:36:19.830271 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:19.830247    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nnb6c_b40eeca6-4402-403d-9029-79ddd8d0e073/kube-rbac-proxy/0.log"
Sep 07 09:36:19.848343 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:19.848322    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nnb6c_b40eeca6-4402-403d-9029-79ddd8d0e073/init-textfile/0.log"
Sep 07 09:36:22.176820 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:22.176794    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-dbgqp_2584a748-e024-4375-9635-2a9a21590230/networking-console-plugin/0.log"
Sep 07 09:36:22.783141 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:22.783111    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/2.log"
Sep 07 09:36:22.787237 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:22.787215    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-hm7ht_b61be2b6-3472-4523-95d6-c964c440fe56/console-operator/3.log"
Sep 07 09:36:23.836871 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:23.836841    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"]
Sep 07 09:36:23.837219 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:23.837199    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="203ec927-f760-4774-9db1-a01f74422f08" containerName="maas-api"
Sep 07 09:36:23.837219 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:23.837210    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="203ec927-f760-4774-9db1-a01f74422f08" containerName="maas-api"
Sep 07 09:36:23.837291 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:23.837265    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="203ec927-f760-4774-9db1-a01f74422f08" containerName="maas-api"
Sep 07 09:36:23.840510 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:23.840493    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:23.842277 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:23.842250    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b5sms\"/\"openshift-service-ca.crt\""
Sep 07 09:36:23.842277 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:23.842269    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b5sms\"/\"default-dockercfg-xmpz2\""
Sep 07 09:36:23.842738 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:23.842724    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b5sms\"/\"kube-root-ca.crt\""
Sep 07 09:36:23.850041 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:23.850017    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"]
Sep 07 09:36:23.973430 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:23.973394    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xdnd\" (UniqueName: \"kubernetes.io/projected/e422d907-ccf6-4e77-a066-53cb11959fda-kube-api-access-2xdnd\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:23.973582 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:23.973435    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e422d907-ccf6-4e77-a066-53cb11959fda-sys\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:23.973582 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:23.973453    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e422d907-ccf6-4e77-a066-53cb11959fda-proc\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:23.973673 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:23.973584    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e422d907-ccf6-4e77-a066-53cb11959fda-lib-modules\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:23.973673 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:23.973636    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e422d907-ccf6-4e77-a066-53cb11959fda-podres\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:24.000315 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.000294    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-9znqs_c2c8a9a7-d1f6-48de-930c-c3250235f861/volume-data-source-validator/0.log"
Sep 07 09:36:24.074966 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.074943    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e422d907-ccf6-4e77-a066-53cb11959fda-lib-modules\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:24.075063 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.074985    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e422d907-ccf6-4e77-a066-53cb11959fda-podres\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:24.075063 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.075016    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xdnd\" (UniqueName: \"kubernetes.io/projected/e422d907-ccf6-4e77-a066-53cb11959fda-kube-api-access-2xdnd\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:24.075063 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.075035    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e422d907-ccf6-4e77-a066-53cb11959fda-sys\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:24.075063 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.075048    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e422d907-ccf6-4e77-a066-53cb11959fda-proc\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:24.075259 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.075112    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e422d907-ccf6-4e77-a066-53cb11959fda-proc\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:24.075259 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.075121    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e422d907-ccf6-4e77-a066-53cb11959fda-lib-modules\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:24.075259 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.075120    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e422d907-ccf6-4e77-a066-53cb11959fda-podres\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:24.075259 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.075124    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e422d907-ccf6-4e77-a066-53cb11959fda-sys\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:24.082563 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.082539    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xdnd\" (UniqueName: \"kubernetes.io/projected/e422d907-ccf6-4e77-a066-53cb11959fda-kube-api-access-2xdnd\") pod \"perf-node-gather-daemonset-6wghf\" (UID: \"e422d907-ccf6-4e77-a066-53cb11959fda\") " pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:24.150586 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.150535    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:24.419145 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.419118    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"]
Sep 07 09:36:24.421167 ip-10-0-141-81 kubenswrapper[2574]: W0907 09:36:24.421142    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode422d907_ccf6_4e77_a066_53cb11959fda.slice/crio-8fd9da0066e19fdee88f4fd666f699c76675f13ecff8fc0fca30b4ce58cadfc2 WatchSource:0}: Error finding container 8fd9da0066e19fdee88f4fd666f699c76675f13ecff8fc0fca30b4ce58cadfc2: Status 404 returned error can't find the container with id 8fd9da0066e19fdee88f4fd666f699c76675f13ecff8fc0fca30b4ce58cadfc2
Sep 07 09:36:24.869000 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.868910    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf" event={"ID":"e422d907-ccf6-4e77-a066-53cb11959fda","Type":"ContainerStarted","Data":"bfc9fc5dbbf21973157dc10dd2c419e5fd7ab745052931cb1bc788eb955da287"}
Sep 07 09:36:24.869000 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.868997    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf" event={"ID":"e422d907-ccf6-4e77-a066-53cb11959fda","Type":"ContainerStarted","Data":"8fd9da0066e19fdee88f4fd666f699c76675f13ecff8fc0fca30b4ce58cadfc2"}
Sep 07 09:36:24.869381 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.869027    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:24.885807 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:24.885760    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf" podStartSLOduration=1.8857449819999998 podStartE2EDuration="1.885744982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 09:36:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 09:36:24.883475944 +0000 UTC m=+2267.847392162" watchObservedRunningTime="2026-09-07 09:36:24.885744982 +0000 UTC m=+2267.849661203"
Sep 07 09:36:25.178564 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:25.178535    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7g5t9_2e215d11-e0ca-493d-a261-35932fbea87c/dns/0.log"
Sep 07 09:36:25.195103 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:25.195077    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7g5t9_2e215d11-e0ca-493d-a261-35932fbea87c/kube-rbac-proxy/0.log"
Sep 07 09:36:25.340425 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:25.340405    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rpdbr_40ddcfef-57f6-46fd-8d22-d4d0b5da9647/dns-node-resolver/0.log"
Sep 07 09:36:25.860264 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:25.860227    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4s9pv_fa333c43-6b4b-4023-8665-bd82f6943f20/node-ca/0.log"
Sep 07 09:36:27.127635 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:27.127613    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b86566b9f-kmwb2_9ec5585a-b1fb-4a96-b88a-22dcbaa6baad/kube-auth-proxy/0.log"
Sep 07 09:36:27.257969 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:27.257937    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54ff77667d-jgqz5_9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b/router/0.log"
Sep 07 09:36:28.094369 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:28.094341    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hzt78_c82b73b1-7d84-4d8f-be08-c7cf655fe39d/serve-healthcheck-canary/0.log"
Sep 07 09:36:28.555810 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:28.555781    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-4ggpx_930228a4-40c6-4a1b-a2a1-599f1de15ec3/insights-operator/0.log"
Sep 07 09:36:28.556492 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:28.556472    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-4ggpx_930228a4-40c6-4a1b-a2a1-599f1de15ec3/insights-operator/1.log"
Sep 07 09:36:28.693813 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:28.693790    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pn9pp_ba613bd6-8f5b-4edc-8a06-b28348288069/kube-rbac-proxy/0.log"
Sep 07 09:36:28.709394 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:28.709374    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pn9pp_ba613bd6-8f5b-4edc-8a06-b28348288069/exporter/0.log"
Sep 07 09:36:28.726264 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:28.726243    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pn9pp_ba613bd6-8f5b-4edc-8a06-b28348288069/extractor/0.log"
Sep 07 09:36:30.796874 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:30.796847    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-rw26k_16652170-c20c-4344-9e2c-55d0f861edc2/manager/0.log"
Sep 07 09:36:30.883800 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:30.883771    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b5sms/perf-node-gather-daemonset-6wghf"
Sep 07 09:36:31.349127 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:31.349092    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c59b5944-z2bls_c077dde0-13cf-4351-a65a-14cb61f02fec/manager/0.log"
Sep 07 09:36:31.363164 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:31.363142    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78c59b5944-z2bls_c077dde0-13cf-4351-a65a-14cb61f02fec/copy-manifests/0.log"
Sep 07 09:36:34.205782 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:34.205753    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b86566b9f-kmwb2_9ec5585a-b1fb-4a96-b88a-22dcbaa6baad/kube-auth-proxy/0.log"
Sep 07 09:36:34.294948 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:34.294907    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54ff77667d-jgqz5_9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b/router/0.log"
Sep 07 09:36:36.160780 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:36.160749    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b86566b9f-kmwb2_9ec5585a-b1fb-4a96-b88a-22dcbaa6baad/kube-auth-proxy/0.log"
Sep 07 09:36:36.265453 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:36.265418    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54ff77667d-jgqz5_9c64f4f0-c3fb-4aed-87a4-b88b17d62a0b/router/0.log"
Sep 07 09:36:37.680329 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:37.680302    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-2hqr9_28974197-205e-48ac-abbb-b57111693793/migrator/0.log"
Sep 07 09:36:37.696314 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:37.696282    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-2hqr9_28974197-205e-48ac-abbb-b57111693793/graceful-termination/0.log"
Sep 07 09:36:38.042647 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:38.042569    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-cmlg4_f96ca98e-8ab7-4335-b49f-c97435d6eab7/kube-storage-version-migrator-operator/1.log"
Sep 07 09:36:38.043495 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:38.043477    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-cmlg4_f96ca98e-8ab7-4335-b49f-c97435d6eab7/kube-storage-version-migrator-operator/0.log"
Sep 07 09:36:39.175891 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:39.175864    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tjrhl_99a03c01-5968-408a-a4ca-957acdce1bd7/kube-multus-additional-cni-plugins/0.log"
Sep 07 09:36:39.191375 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:39.191354    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tjrhl_99a03c01-5968-408a-a4ca-957acdce1bd7/egress-router-binary-copy/0.log"
Sep 07 09:36:39.207015 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:39.206995    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tjrhl_99a03c01-5968-408a-a4ca-957acdce1bd7/cni-plugins/0.log"
Sep 07 09:36:39.223909 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:39.223894    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tjrhl_99a03c01-5968-408a-a4ca-957acdce1bd7/bond-cni-plugin/0.log"
Sep 07 09:36:39.238857 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:39.238842    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tjrhl_99a03c01-5968-408a-a4ca-957acdce1bd7/routeoverride-cni/0.log"
Sep 07 09:36:39.253451 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:39.253434    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tjrhl_99a03c01-5968-408a-a4ca-957acdce1bd7/whereabouts-cni-bincopy/0.log"
Sep 07 09:36:39.269186 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:39.269170    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tjrhl_99a03c01-5968-408a-a4ca-957acdce1bd7/whereabouts-cni/0.log"
Sep 07 09:36:39.301853 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:39.301833    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-l8xfv_23a08429-ee73-45ad-8a29-bbcf1a359f02/kube-multus/0.log"
Sep 07 09:36:39.438729 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:39.438661    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-c594r_2aee0981-afb8-47c7-8484-142812fc6d42/network-metrics-daemon/0.log"
Sep 07 09:36:39.452172 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:39.452151    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-c594r_2aee0981-afb8-47c7-8484-142812fc6d42/kube-rbac-proxy/0.log"
Sep 07 09:36:40.622726 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:40.622625    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-controller/0.log"
Sep 07 09:36:40.634829 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:40.634802    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/0.log"
Sep 07 09:36:40.635071 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:40.635056    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovn-acl-logging/1.log"
Sep 07 09:36:40.651842 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:40.651821    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/kube-rbac-proxy-node/0.log"
Sep 07 09:36:40.667918 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:40.667901    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/kube-rbac-proxy-ovn-metrics/0.log"
Sep 07 09:36:40.684276 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:40.684256    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/northd/0.log"
Sep 07 09:36:40.699587 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:40.699569    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/nbdb/0.log"
Sep 07 09:36:40.716319 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:40.716303    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/sbdb/0.log"
Sep 07 09:36:40.762607 ip-10-0-141-81 kubenswrapper[2574]: I0907 09:36:40.762582    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zdlqw_eebe8c71-1cd7-4040-af2d-7b636fd884c2/ovnkube-controller/0.log"