Sep 19 10:34:37.632796 ip-10-0-140-105 systemd[1]: Starting Kubernetes Kubelet...
Sep 19 10:34:40.925648 ip-10-0-140-105 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 10:34:40.925648 ip-10-0-140-105 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 19 10:34:40.925648 ip-10-0-140-105 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 10:34:40.925648 ip-10-0-140-105 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 19 10:34:40.925648 ip-10-0-140-105 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 10:34:40.926408 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.925714    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 19 10:34:40.929547 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929532    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 10:34:40.929547 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929547    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929551    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929554    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929557    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929560    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929563    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929566    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929569    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929572    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929575    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929578    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929581    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929589    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929592    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929595    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929598    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929601    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929603    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929606    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 10:34:40.929616 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929609    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929612    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929614    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929617    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929620    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929623    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929626    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929629    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929632    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929635    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929637    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929640    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929642    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929645    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929648    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929650    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929653    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929655    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929658    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929661    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 10:34:40.930057 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929664    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929667    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929669    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929672    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929674    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929678    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929680    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929683    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929685    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929688    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929691    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929694    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929696    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929699    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929702    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929705    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929708    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929710    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929713    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929716    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 10:34:40.930560 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929719    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929721    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929724    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929727    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929729    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929732    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929735    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929737    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929741    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929743    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929746    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929750    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929754    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929757    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929760    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929765    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929767    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929770    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929772    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 10:34:40.931044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929775    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929777    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929780    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929783    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929785    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929788    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929791    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929794    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929797    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929799    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929802    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929805    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929808    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929810    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929813    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929816    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929819    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929823    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929826    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 10:34:40.931496 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.929830    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 10:34:40.932117 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932104    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 10:34:40.932117 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932117    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932121    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932124    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932128    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932131    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932134    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932137    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932140    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932142    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932145    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932148    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932150    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932153    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932156    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932158    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932161    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932164    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932167    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932169    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932172    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 10:34:40.932169 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932175    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932178    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932181    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932184    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932186    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932189    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932192    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932194    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932197    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932199    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932202    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932205    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932207    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932210    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932212    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932215    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932218    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932220    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932223    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 10:34:40.932676 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932226    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932229    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932232    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932234    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932237    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932239    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932242    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932246    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932248    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932251    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932253    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932256    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932258    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932261    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932263    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932268    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932271    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932274    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932276    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 10:34:40.933137 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932280    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932284    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932288    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932291    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932294    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932297    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932300    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932302    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932307    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932311    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932314    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932317    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932320    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932322    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932325    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932328    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932331    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932333    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932336    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 10:34:40.933630 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932338    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932341    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932343    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932346    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932349    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932351    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932354    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932356    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932359    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932363    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932365    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932368    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932371    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932373    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932376    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932379    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932381    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932384    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932387    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932389    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 10:34:40.934114 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932392    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932757    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932762    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932765    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932767    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932770    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932773    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932776    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932779    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932782    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932785    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932787    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932790    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932793    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932796    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932799    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932801    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932804    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932808    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932811    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 10:34:40.934659 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932814    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932818    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932821    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932824    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932826    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932829    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932832    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932834    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932839    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932842    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932845    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932848    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932851    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932853    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932856    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932859    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932861    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932864    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932866    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 10:34:40.935130 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932869    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932872    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932875    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932877    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932882    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932885    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932888    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932891    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932894    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932898    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932901    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932903    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932906    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932909    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932913    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932916    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932919    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932922    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932925    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 10:34:40.935632 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932928    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932930    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932933    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932935    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932938    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932940    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932943    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932946    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932948    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932951    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932953    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932956    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932958    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932961    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932964    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932967    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932970    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932972    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932975    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932977    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 10:34:40.936096 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932980    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932982    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932985    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932987    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932990    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932992    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932994    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.932998    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933000    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933003    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933006    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933008    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933011    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933014    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933016    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933019    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933021    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933024    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933026    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933029    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 10:34:40.936578 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933031    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933034    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933106    2579 flags.go:64] FLAG: --address="0.0.0.0"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933147    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933154    2579 flags.go:64] FLAG: --anonymous-auth="true"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933158    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933162    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933165    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933170    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933174    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933178    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933181    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933185    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933188    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933191    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933194    2579 flags.go:64] FLAG: --cgroup-root=""
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933197    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933201    2579 flags.go:64] FLAG: --client-ca-file=""
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933204    2579 flags.go:64] FLAG: --cloud-provider="external"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933207    2579 flags.go:64] FLAG: --cluster-dns="[]"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933212    2579 flags.go:64] FLAG: --cluster-domain=""
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933215    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933218    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933221    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 19 10:34:40.937091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933224    2579 flags.go:64] FLAG: --container-log-max-files="5"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933228    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933231    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933234    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933238    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933241    2579 flags.go:64] FLAG: --contention-profiling="false"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933244    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933247    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933250    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933253    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933257    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933260    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933263    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933266    2579 flags.go:64] FLAG: --enable-load-reader="false"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933269    2579 flags.go:64] FLAG: --enable-server="true"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933272    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933277    2579 flags.go:64] FLAG: --event-burst="100"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933281    2579 flags.go:64] FLAG: --event-qps="50"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933284    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933287    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933290    2579 flags.go:64] FLAG: --eviction-hard=""
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933294    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933297    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933300    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933303    2579 flags.go:64] FLAG: --eviction-soft=""
Sep 19 10:34:40.937665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933306    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933309    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933312    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933315    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933318    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933321    2579 flags.go:64] FLAG: --fail-swap-on="true"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933324    2579 flags.go:64] FLAG: --feature-gates=""
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933328    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933331    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933334    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933338    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933341    2579 flags.go:64] FLAG: --healthz-port="10248"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933344    2579 flags.go:64] FLAG: --help="false"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933346    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-140-105.ec2.internal"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933350    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933352    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933355    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933359    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933362    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933365    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933368    2579 flags.go:64] FLAG: --image-service-endpoint=""
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933371    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933373    2579 flags.go:64] FLAG: --kube-api-burst="100"
Sep 19 10:34:40.938261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933376    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933379    2579 flags.go:64] FLAG: --kube-api-qps="50"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933383    2579 flags.go:64] FLAG: --kube-reserved=""
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933386    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933389    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933392    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933395    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933398    2579 flags.go:64] FLAG: --lock-file=""
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933400    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933403    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933406    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933412    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933415    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933418    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933421    2579 flags.go:64] FLAG: --logging-format="text"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933423    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933427    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933430    2579 flags.go:64] FLAG: --manifest-url=""
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933433    2579 flags.go:64] FLAG: --manifest-url-header=""
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933437    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933440    2579 flags.go:64] FLAG: --max-open-files="1000000"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933443    2579 flags.go:64] FLAG: --max-pods="110"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933446    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933450    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933453    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 19 10:34:40.938863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933455    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933458    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933461    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933464    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933471    2579 flags.go:64] FLAG: --node-status-max-images="50"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933474    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933477    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933480    2579 flags.go:64] FLAG: --pod-cidr=""
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933483    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933489    2579 flags.go:64] FLAG: --pod-manifest-path=""
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933492    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933495    2579 flags.go:64] FLAG: --pods-per-core="0"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933498    2579 flags.go:64] FLAG: --port="10250"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933501    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933504    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a68a7b153c545965"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933508    2579 flags.go:64] FLAG: --qos-reserved=""
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933510    2579 flags.go:64] FLAG: --read-only-port="10255"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933514    2579 flags.go:64] FLAG: --register-node="true"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933519    2579 flags.go:64] FLAG: --register-with-taints=""
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933535    2579 flags.go:64] FLAG: --registry-burst="10"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933538    2579 flags.go:64] FLAG: --registry-qps="5"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933541    2579 flags.go:64] FLAG: --reserved-cpus=""
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933544    2579 flags.go:64] FLAG: --reserved-memory=""
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933548    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933551    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 19 10:34:40.939464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933554    2579 flags.go:64] FLAG: --rotate-certificates="false"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933557    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933560    2579 flags.go:64] FLAG: --runonce="false"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933563    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933566    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933570    2579 flags.go:64] FLAG: --seccomp-default="false"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933573    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933576    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933579    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933582    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933587    2579 flags.go:64] FLAG: --storage-driver-password="root"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933590    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933593    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933596    2579 flags.go:64] FLAG: --storage-driver-user="root"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933599    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933602    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933605    2579 flags.go:64] FLAG: --system-cgroups=""
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933608    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933613    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933617    2579 flags.go:64] FLAG: --tls-cert-file=""
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933620    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933624    2579 flags.go:64] FLAG: --tls-min-version=""
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933627    2579 flags.go:64] FLAG: --tls-private-key-file=""
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933630    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933633    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 19 10:34:40.940453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933638    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933641    2579 flags.go:64] FLAG: --v="2"
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933645    2579 flags.go:64] FLAG: --version="false"
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933649    2579 flags.go:64] FLAG: --vmodule=""
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933652    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.933655    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933766    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933770    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933773    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933776    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933779    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933782    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933786    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933788    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933791    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933794    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933797    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933801    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933804    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933807    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933809    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 10:34:40.941597 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933812    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933814    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933817    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933819    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933822    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933825    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933828    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933831    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933833    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933838    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933841    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933845    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933848    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933851    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933854    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933864    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933867    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933870    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933873    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 10:34:40.942468 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933876    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933879    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933882    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933885    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933888    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933891    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933894    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933897    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933900    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933904    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933906    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933911    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933914    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933917    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933920    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933922    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933925    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933928    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933931    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 10:34:40.943276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933934    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933937    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933940    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933942    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933945    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933949    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933952    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933955    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933957    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933960    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933963    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933966    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933969    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933971    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933974    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933977    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933979    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933982    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933986    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933989    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 10:34:40.944118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933992    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933994    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.933999    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934001    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934004    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934007    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934009    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934012    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934015    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934017    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934020    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934023    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934025    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934029    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934032    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934034    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934037    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934041    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934044    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934046    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 10:34:40.944747 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934049    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934052    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934054    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934057    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934059    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.934062    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.934567    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.942712    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.942732    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.942760    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.942770    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942865    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942873    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942878    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942883    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 10:34:40.945596 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942888    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942892    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942897    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942901    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942908    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942914    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942919    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942924    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942928    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942933    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942937    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942942    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942946    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942951    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942955    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942960    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942964    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942968    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942972    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 10:34:40.946273 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942977    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942981    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942985    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942989    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942995    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.942999    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943004    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943009    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943013    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943018    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943022    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943026    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943031    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943035    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943040    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943044    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943048    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943052    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943057    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943061    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 10:34:40.946992 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943066    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943070    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943075    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943082    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943088    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943092    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943097    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943102    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943106    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943112    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943117    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943121    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943126    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943130    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943134    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943138    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943143    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943147    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943151    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943156    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 10:34:40.947480 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943160    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943164    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943169    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943173    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943177    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943182    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943186    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943190    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943195    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943199    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943203    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943207    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943211    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943215    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943219    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943224    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943229    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943234    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943238    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943242    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 10:34:40.948042 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943246    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943251    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943255    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943259    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943263    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943267    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943271    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943275    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943279    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943282    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943288    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943294    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943299    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943303    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943307    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 10:34:40.948593 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943312    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.943319    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943503    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943512    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943517    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943539    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943544    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943549    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943553    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943557    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943561    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943567    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943574    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943579    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943584    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 10:34:40.948967 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943589    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943593    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943598    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943602    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943606    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943610    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943615    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943619    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943624    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943630    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943636    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943641    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943645    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943650    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943654    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943658    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943663    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943667    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943671    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943675    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 10:34:40.949365 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943679    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943684    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943688    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943692    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943696    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943700    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943704    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943709    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943713    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943717    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943721    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943726    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943731    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943735    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943739    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943743    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943747    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943751    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943757    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943761    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 10:34:40.950024 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943765    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943769    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943774    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943778    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943782    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943786    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943790    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943794    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943798    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943802    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943806    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943811    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943815    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943819    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943823    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943833    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943837    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943842    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943846    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943850    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 10:34:40.950836 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943854    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943858    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943862    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943866    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943871    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943877    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943881    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943886    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943890    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943894    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943898    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943905    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943910    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943914    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943919    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943924    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943928    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943933    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943937    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 10:34:40.951706 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943941    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 10:34:40.952474 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943945    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 10:34:40.952474 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943949    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 10:34:40.952474 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943954    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 10:34:40.952474 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943957    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 10:34:40.952474 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943962    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 10:34:40.952474 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:40.943966    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 10:34:40.952474 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.943974    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 10:34:40.952474 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.944156    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 19 10:34:40.952474 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.946764    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 19 10:34:40.952474 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.947661    2579 server.go:1019] "Starting client certificate rotation"
Sep 19 10:34:40.952474 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.947767    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 10:34:40.952474 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.947803    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 10:34:40.952474 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.950293    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 10:34:40.952920 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.952115    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 10:34:40.955906 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.955889    2579 log.go:25] "Validated CRI v1 runtime API"
Sep 19 10:34:40.957385 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.957370    2579 log.go:25] "Validated CRI v1 image API"
Sep 19 10:34:40.958237 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.958220    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 19 10:34:40.961323 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.961302    2579 fs.go:135] Filesystem UUIDs: map[4911706a-a4fd-40e4-84d5-cfa92aae1168:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b831ac4e-d09f-4298-beef-7282bce56609:/dev/nvme0n1p3]
Sep 19 10:34:40.961394 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.961322    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 19 10:34:40.966401 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.966289    2579 manager.go:216] Machine: {Timestamp:2026-09-19 10:34:40.965828017 +0000 UTC m=+1.877521526 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098980 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec288885f420f79830b372f6641e1a3c SystemUUID:ec288885-f420-f798-30b3-72f6641e1a3c BootID:716498f7-d6dc-4e38-921e-b89893e53b4a Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f2:56:70:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f2:56:70:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:ae:66:65:e0:46 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 19 10:34:40.966401 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.966392    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 19 10:34:40.966542 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.966490    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 19 10:34:40.967847 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.967815    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 19 10:34:40.968015 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.967849    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-105.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 19 10:34:40.968874 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.968861    2579 topology_manager.go:138] "Creating topology manager with none policy"
Sep 19 10:34:40.968941 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.968880    2579 container_manager_linux.go:309] "Creating device plugin manager"
Sep 19 10:34:40.968941 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.968890    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 10:34:40.968941 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.968907    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 10:34:40.970004 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.969991    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 19 10:34:40.973240 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.973226    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 10:34:40.973367 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.973356    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 19 10:34:40.973436 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.973411    2579 kubelet.go:486] "Attempting to sync node with API server"
Sep 19 10:34:40.973436 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.973430    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 19 10:34:40.973544 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.973451    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 19 10:34:40.973544 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.973465    2579 kubelet.go:398] "Adding apiserver pod source"
Sep 19 10:34:40.973544 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.973477    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 19 10:34:40.976108 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.976085    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 10:34:40.976185 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.976129    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 10:34:40.981092 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.981066    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 10:34:40.982669 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.982651    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 19 10:34:40.984666 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.984653    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 19 10:34:40.984710 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.984682    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 19 10:34:40.985577 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.985567    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 19 10:34:40.985617 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.985582    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 19 10:34:40.985617 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.985588    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 19 10:34:40.985617 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.985593    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 19 10:34:40.985617 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.985602    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 19 10:34:40.985617 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.985607    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 19 10:34:40.985617 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.985620    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 19 10:34:40.985794 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.985627    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 19 10:34:40.985794 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.985634    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 19 10:34:40.985794 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.985639    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 19 10:34:40.985794 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.985647    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 19 10:34:40.985794 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.985652    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 19 10:34:40.985794 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.985677    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 19 10:34:40.985794 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.985683    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 19 10:34:40.989195 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.989183    2579 server.go:1268] "Started kubelet"
Sep 19 10:34:40.989271 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.989237    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 19 10:34:40.989318 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.989262    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 19 10:34:40.989363 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.989331    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 19 10:34:40.990059 ip-10-0-140-105 systemd[1]: Started Kubernetes Kubelet.
Sep 19 10:34:40.990688 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.990264    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 19 10:34:40.991096 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.991037    2579 server.go:310] "Adding debug handlers to kubelet server"
Sep 19 10:34:40.993025 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:40.993005    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 19 10:34:40.993240 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.993226    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-105.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 19 10:34:40.993284 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:40.993267    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 19 10:34:40.993618 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:40.993599    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 19 10:34:40.996360 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.996346    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 19 10:34:40.996408 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.996369    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 19 10:34:40.996452 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:40.996443    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:40.996859 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.996840    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 19 10:34:40.996859 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.996841    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 19 10:34:40.997007 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.996867    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 19 10:34:40.997007 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.997001    2579 reconstruct.go:97] "Volume reconstruction finished"
Sep 19 10:34:40.997091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.997011    2579 reconciler.go:29] "Reconciler: start to sync state"
Sep 19 10:34:40.997169 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.997155    2579 factory.go:55] Registering systemd factory
Sep 19 10:34:40.997214 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.997172    2579 factory.go:223] Registration of the systemd container factory successfully
Sep 19 10:34:40.997426 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.997408    2579 factory.go:153] Registering CRI-O factory
Sep 19 10:34:40.997426 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.997425    2579 factory.go:223] Registration of the crio container factory successfully
Sep 19 10:34:40.997560 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.997474    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 19 10:34:40.997560 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.997496    2579 factory.go:103] Registering Raw factory
Sep 19 10:34:40.997560 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.997557    2579 manager.go:1196] Started watching for new ooms in manager
Sep 19 10:34:40.997928 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:40.997917    2579 manager.go:349] Starting recovery of all containers
Sep 19 10:34:40.999823 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:40.999782    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 19 10:34:41.000041 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.000004    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 19 10:34:41.004705 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.004678    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 19 10:34:41.021859 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.021838    2579 manager.go:354] Recovery completed
Sep 19 10:34:41.026734 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.026721    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:34:41.028289 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.028270    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:34:41.028364 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.028301    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:34:41.028364 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.028312    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:34:41.028903 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.028887    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 19 10:34:41.028903 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.028903    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 19 10:34:41.029016 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.028888    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:34:41.029016 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.028921    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 10:34:41.029204 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.029188    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xmtrk"
Sep 19 10:34:41.029583 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.029569    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:34:41.029641 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.029593    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:34:41.029641 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.029602    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:34:41.031046 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.031032    2579 policy_none.go:49] "None policy: Start"
Sep 19 10:34:41.031094 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.031049    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 19 10:34:41.031094 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.031059    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 19 10:34:41.032961 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.032950    2579 policy_none.go:47] "Start"
Sep 19 10:34:41.036239 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.036222    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xmtrk"
Sep 19 10:34:41.038958 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.038123    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-105.ec2.internal.18d6b26c7f59d928  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-105.ec2.internal,UID:ip-10-0-140-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-105.ec2.internal,},FirstTimestamp:2026-09-19 10:34:40.989157672 +0000 UTC m=+1.900851182,LastTimestamp:2026-09-19 10:34:40.989157672 +0000 UTC m=+1.900851182,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-105.ec2.internal,}"
Sep 19 10:34:41.041787 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.041729    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-105.ec2.internal.18d6b26c81aef2d2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-105.ec2.internal,UID:ip-10-0-140-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-105.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-105.ec2.internal,},FirstTimestamp:2026-09-19 10:34:41.028289234 +0000 UTC m=+1.939982747,LastTimestamp:2026-09-19 10:34:41.028289234 +0000 UTC m=+1.939982747,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-105.ec2.internal,}"
Sep 19 10:34:41.071103 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.071077    2579 manager.go:337] "Starting Device Plugin manager"
Sep 19 10:34:41.084263 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.071113    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 19 10:34:41.084263 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.071125    2579 server.go:87] "Starting device plugin registration server"
Sep 19 10:34:41.084263 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.071328    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 19 10:34:41.084263 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.071344    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 19 10:34:41.084263 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.071413    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 19 10:34:41.084263 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.071477    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 19 10:34:41.084263 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.071485    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 19 10:34:41.084263 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.072007    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 19 10:34:41.084263 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.072058    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:41.108744 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.108724    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 19 10:34:41.108815 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.108767    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 19 10:34:41.108815 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.108785    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 19 10:34:41.108908 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.108816    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 19 10:34:41.112558 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.112540    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 10:34:41.172425 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.172396    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:34:41.173198 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.173181    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:34:41.173282 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.173213    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:34:41.173282 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.173230    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:34:41.173282 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.173257    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.189165 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.189116    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.189165 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.189137    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-105.ec2.internal\": node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:41.209768 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.209748    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"]
Sep 19 10:34:41.209836 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.209801    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:34:41.210549 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.210513    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:34:41.210629 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.210557    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:34:41.210629 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.210567    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:34:41.212777 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.212765    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:34:41.212905 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.212885    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.212942 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.212913    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:34:41.215435 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.215420    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:34:41.215484 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.215424    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:34:41.215484 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.215472    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:34:41.215484 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.215482    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:34:41.215587 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.215451    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:34:41.215587 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.215540    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:34:41.217624 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.217603    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.217697 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.217630    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:34:41.218371 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.218356    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:34:41.218421 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.218381    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:34:41.218421 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.218391    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:34:41.223229 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.223213    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:41.242136 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.242120    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-105.ec2.internal\" not found" node="ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.246297 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.246282    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-105.ec2.internal\" not found" node="ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.298305 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.298284    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ca6b3a31a573fb44477ffc56660c8ae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal\" (UID: \"4ca6b3a31a573fb44477ffc56660c8ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.298376 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.298307    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b8679eb2ddf3de7904fb5919a04512a3-config\") pod \"kube-apiserver-proxy-ip-10-0-140-105.ec2.internal\" (UID: \"b8679eb2ddf3de7904fb5919a04512a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.298376 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.298325    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4ca6b3a31a573fb44477ffc56660c8ae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal\" (UID: \"4ca6b3a31a573fb44477ffc56660c8ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.323533 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.323501    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:41.398626 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.398602    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4ca6b3a31a573fb44477ffc56660c8ae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal\" (UID: \"4ca6b3a31a573fb44477ffc56660c8ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.398708 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.398629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ca6b3a31a573fb44477ffc56660c8ae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal\" (UID: \"4ca6b3a31a573fb44477ffc56660c8ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.398708 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.398647    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b8679eb2ddf3de7904fb5919a04512a3-config\") pod \"kube-apiserver-proxy-ip-10-0-140-105.ec2.internal\" (UID: \"b8679eb2ddf3de7904fb5919a04512a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.398708 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.398672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b8679eb2ddf3de7904fb5919a04512a3-config\") pod \"kube-apiserver-proxy-ip-10-0-140-105.ec2.internal\" (UID: \"b8679eb2ddf3de7904fb5919a04512a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.398708 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.398695    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4ca6b3a31a573fb44477ffc56660c8ae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal\" (UID: \"4ca6b3a31a573fb44477ffc56660c8ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.398867 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.398711    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ca6b3a31a573fb44477ffc56660c8ae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal\" (UID: \"4ca6b3a31a573fb44477ffc56660c8ae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.424222 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.424202    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:41.524301 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.524244    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:41.545403 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.545385    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.549286 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.549273    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:41.624895 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.624743    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:41.725223 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.725199    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:41.825690 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.825625    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:41.926179 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:41.926144    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:41.947300 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.947279    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 19 10:34:41.947441 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:41.947425    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 10:34:42.026467 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:42.026442    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:42.040582 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.040540    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-18 10:29:41 +0000 UTC" deadline="2028-06-08 23:22:40.355493174 +0000 UTC"
Sep 19 10:34:42.040582 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.040579    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15084h47m58.314917159s"
Sep 19 10:34:42.110381 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.110310    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 10:34:42.114637 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.114620    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 10:34:42.115549 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.115535    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 10:34:42.115609 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.115564    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 10:34:42.115609 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.115575    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeHasSufficientPID"
Sep 19 10:34:42.126802 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:42.126784    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:42.169340 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:42.169314    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca6b3a31a573fb44477ffc56660c8ae.slice/crio-ce8f6fa25b9962b5b519009da8e6644af2c8204c11a7a7b6ea6a7de46ee48ece WatchSource:0}: Error finding container ce8f6fa25b9962b5b519009da8e6644af2c8204c11a7a7b6ea6a7de46ee48ece: Status 404 returned error can't find the container with id ce8f6fa25b9962b5b519009da8e6644af2c8204c11a7a7b6ea6a7de46ee48ece
Sep 19 10:34:42.169682 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:42.169668    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8679eb2ddf3de7904fb5919a04512a3.slice/crio-0b92c3f24313817b2ba1fde9281331ca4adc05f6139aa484b6a0d8eb79721484 WatchSource:0}: Error finding container 0b92c3f24313817b2ba1fde9281331ca4adc05f6139aa484b6a0d8eb79721484: Status 404 returned error can't find the container with id 0b92c3f24313817b2ba1fde9281331ca4adc05f6139aa484b6a0d8eb79721484
Sep 19 10:34:42.175187 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.175172    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 10:34:42.227612 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:42.227589    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:42.328179 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:42.328148    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:42.428635 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:42.428564    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-105.ec2.internal\" not found"
Sep 19 10:34:42.504327 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.504300    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 10:34:42.543794 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.543764    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 10:34:42.597179 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.597140    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:42.608722 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.608614    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 10:34:42.609867 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.609698    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"
Sep 19 10:34:42.616869 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.616850    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 10:34:42.975364 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.975338    2579 apiserver.go:52] "Watching apiserver"
Sep 19 10:34:42.986042 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.986016    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 19 10:34:42.986437 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.986401    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk","openshift-dns/node-resolver-wr84k","openshift-multus/multus-8vvd2","openshift-multus/network-metrics-daemon-lnqqz","openshift-network-diagnostics/network-check-target-9f2kq","openshift-cluster-node-tuning-operator/tuned-cl9lw","openshift-image-registry/node-ca-qrxfh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal","openshift-multus/multus-additional-cni-plugins-p4shs","openshift-network-operator/iptables-alerter-xchfp","openshift-ovn-kubernetes/ovnkube-node-wzspq","kube-system/global-pull-secret-syncer-vr6wb","kube-system/konnectivity-agent-5xl48","kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal"]
Sep 19 10:34:42.988795 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.988775    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qrxfh"
Sep 19 10:34:42.990791 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.990774    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5xl48"
Sep 19 10:34:42.992396 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.992381    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 19 10:34:42.992486 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.992391    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 19 10:34:42.992486 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.992392    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 19 10:34:42.992611 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.992501    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7sn6x\""
Sep 19 10:34:42.993353 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.993336    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 19 10:34:42.993574 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.993556    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 19 10:34:42.993753 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.993739    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4srmw\""
Sep 19 10:34:42.993940 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.993923    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:42.996010 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.995989    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:42.996100 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:42.996078    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:34:42.996480 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.996458    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 19 10:34:42.996611 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.996500    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 19 10:34:42.996670 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.996467    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-n5hzt\""
Sep 19 10:34:42.996827 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.996808    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 19 10:34:42.996916 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.996867    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 19 10:34:42.997157 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.997014    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 19 10:34:42.998632 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:42.998613    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:42.998736 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:42.998672    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:34:43.000810 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.000789    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.003103 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.003083    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.003600 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.003581    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-clnx7\""
Sep 19 10:34:43.003696 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.003598    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 19 10:34:43.003696 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.003675    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 19 10:34:43.005308 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.005289    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.005844 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.005722    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-878zs\" (UniqueName: \"kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs\") pod \"network-check-target-9f2kq\" (UID: \"4c9f2fc8-10e3-4bda-8b12-8f05918712f2\") " pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:43.005844 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.005756    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-multus-cni-dir\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.005844 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.005785    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-run-netns\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.005844 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.005811    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-var-lib-cni-bin\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.006090 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.005853    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-etc-kubernetes\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.006090 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.005902    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs\") pod \"network-metrics-daemon-lnqqz\" (UID: \"f85be02b-a097-4f90-844c-21d54e7dc53d\") " pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:43.006090 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.005933    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2c21f7fa-a5d2-4c47-9e12-7ee4a905e870-serviceca\") pod \"node-ca-qrxfh\" (UID: \"2c21f7fa-a5d2-4c47-9e12-7ee4a905e870\") " pod="openshift-image-registry/node-ca-qrxfh"
Sep 19 10:34:43.006090 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.005958    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzrb5\" (UniqueName: \"kubernetes.io/projected/2c21f7fa-a5d2-4c47-9e12-7ee4a905e870-kube-api-access-tzrb5\") pod \"node-ca-qrxfh\" (UID: \"2c21f7fa-a5d2-4c47-9e12-7ee4a905e870\") " pod="openshift-image-registry/node-ca-qrxfh"
Sep 19 10:34:43.006090 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.005982    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-multus-socket-dir-parent\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.006090 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006005    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-run-k8s-cni-cncf-io\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.006090 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006029    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-run-multus-certs\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.006090 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006051    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv6b2\" (UniqueName: \"kubernetes.io/projected/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-kube-api-access-fv6b2\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.006090 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006074    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c21f7fa-a5d2-4c47-9e12-7ee4a905e870-host\") pod \"node-ca-qrxfh\" (UID: \"2c21f7fa-a5d2-4c47-9e12-7ee4a905e870\") " pod="openshift-image-registry/node-ca-qrxfh"
Sep 19 10:34:43.006584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006098    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/347ba7f0-42f9-43c2-b54f-55466a818064-agent-certs\") pod \"konnectivity-agent-5xl48\" (UID: \"347ba7f0-42f9-43c2-b54f-55466a818064\") " pod="kube-system/konnectivity-agent-5xl48"
Sep 19 10:34:43.006584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006122    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-cnibin\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.006584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-var-lib-cni-multus\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.006584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006183    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-var-lib-kubelet\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.006584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006204    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-hostroot\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.006584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006223    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-multus-conf-dir\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.006584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006231    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 19 10:34:43.006584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006259    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/347ba7f0-42f9-43c2-b54f-55466a818064-konnectivity-ca\") pod \"konnectivity-agent-5xl48\" (UID: \"347ba7f0-42f9-43c2-b54f-55466a818064\") " pod="kube-system/konnectivity-agent-5xl48"
Sep 19 10:34:43.006584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006283    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-system-cni-dir\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.006584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-cni-binary-copy\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.006584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006328    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-multus-daemon-config\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.006584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006351    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8wmw\" (UniqueName: \"kubernetes.io/projected/f85be02b-a097-4f90-844c-21d54e7dc53d-kube-api-access-j8wmw\") pod \"network-metrics-daemon-lnqqz\" (UID: \"f85be02b-a097-4f90-844c-21d54e7dc53d\") " pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:43.007160 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.006596    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sh9pb\""
Sep 19 10:34:43.007976 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.007281    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 19 10:34:43.007976 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.007724    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 19 10:34:43.007976 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.007776    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-npkq5\""
Sep 19 10:34:43.008182 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.007985    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wr84k"
Sep 19 10:34:43.010182 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.010164    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xchfp"
Sep 19 10:34:43.013022 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.012697    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.013820 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.013800    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 19 10:34:43.014720 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.014686    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 19 10:34:43.014806 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.014750    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 19 10:34:43.015071 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.015055    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 19 10:34:43.015138 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.015125    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:43.015196 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.015178    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:34:43.015311 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.015296    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-v2rfc\""
Sep 19 10:34:43.016382 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.016364    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 19 10:34:43.018100 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.017730    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 19 10:34:43.018100 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.017866    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 19 10:34:43.018100 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.017957    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4j4jp\""
Sep 19 10:34:43.018100 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.018023    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 19 10:34:43.018784 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.018768    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 19 10:34:43.020566 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.019075    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 19 10:34:43.020566 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.019269    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 19 10:34:43.020566 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.019278    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 19 10:34:43.020969 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.020950    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p6sdr\""
Sep 19 10:34:43.021158 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.021140    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 19 10:34:43.030288 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.030071    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 10:34:43.052333 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.052310    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qshh7"
Sep 19 10:34:43.063812 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.063795    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qshh7"
Sep 19 10:34:43.098277 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.098257    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 19 10:34:43.106837 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.106814    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e4730dd7-e761-4497-bf79-2b72d0828b8f-iptables-alerter-script\") pod \"iptables-alerter-xchfp\" (UID: \"e4730dd7-e761-4497-bf79-2b72d0828b8f\") " pod="openshift-network-operator/iptables-alerter-xchfp"
Sep 19 10:34:43.106998 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.106853    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-var-lib-openvswitch\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.106998 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.106880    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-cni-bin\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.106998 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.106904    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:43.106998 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.106927    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-host\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.106998 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.106949    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-tmp\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.106998 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.106973    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-multus-cni-dir\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.106998 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.106987    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-run-netns\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.106998 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107002    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-systemd-units\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.107384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107022    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-run-ovn\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.107384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107043    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-run-ovn-kubernetes\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.107384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107062    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd50bc98-a748-4587-aacf-56daf296fe88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.107384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107079    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5492514c-c9bf-4516-9314-d1adda92b9ff-tmp-dir\") pod \"node-resolver-wr84k\" (UID: \"5492514c-c9bf-4516-9314-d1adda92b9ff\") " pod="openshift-dns/node-resolver-wr84k"
Sep 19 10:34:43.107384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107099    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzrb5\" (UniqueName: \"kubernetes.io/projected/2c21f7fa-a5d2-4c47-9e12-7ee4a905e870-kube-api-access-tzrb5\") pod \"node-ca-qrxfh\" (UID: \"2c21f7fa-a5d2-4c47-9e12-7ee4a905e870\") " pod="openshift-image-registry/node-ca-qrxfh"
Sep 19 10:34:43.107384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107152    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-run-k8s-cni-cncf-io\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.107384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107172    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv6b2\" (UniqueName: \"kubernetes.io/projected/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-kube-api-access-fv6b2\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.107384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86c4v\" (UniqueName: \"kubernetes.io/projected/e4730dd7-e761-4497-bf79-2b72d0828b8f-kube-api-access-86c4v\") pod \"iptables-alerter-xchfp\" (UID: \"e4730dd7-e761-4497-bf79-2b72d0828b8f\") " pod="openshift-network-operator/iptables-alerter-xchfp"
Sep 19 10:34:43.107384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107222    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-sysconfig\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.107384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107227    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-multus-cni-dir\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.107384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107244    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-sysctl-d\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.107384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c21f7fa-a5d2-4c47-9e12-7ee4a905e870-host\") pod \"node-ca-qrxfh\" (UID: \"2c21f7fa-a5d2-4c47-9e12-7ee4a905e870\") " pod="openshift-image-registry/node-ca-qrxfh"
Sep 19 10:34:43.107384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107342    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c21f7fa-a5d2-4c47-9e12-7ee4a905e870-host\") pod \"node-ca-qrxfh\" (UID: \"2c21f7fa-a5d2-4c47-9e12-7ee4a905e870\") " pod="openshift-image-registry/node-ca-qrxfh"
Sep 19 10:34:43.108012 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107409    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-run-netns\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.108012 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107846    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/347ba7f0-42f9-43c2-b54f-55466a818064-agent-certs\") pod \"konnectivity-agent-5xl48\" (UID: \"347ba7f0-42f9-43c2-b54f-55466a818064\") " pod="kube-system/konnectivity-agent-5xl48"
Sep 19 10:34:43.108012 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107910    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-run-k8s-cni-cncf-io\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.108012 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.107925    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-var-lib-kubelet\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.108202 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108048    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-hostroot\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.108202 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108069    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-var-lib-kubelet\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.108202 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108113    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ba18fe0c-ca67-4702-8160-e4490b615f24-ovnkube-script-lib\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.108202 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108141    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-hostroot\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.108202 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108175    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/630f71c6-b3e9-4935-8c99-921f92d7494c-kubelet-config\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:43.108428 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd50bc98-a748-4587-aacf-56daf296fe88-cnibin\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.108428 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108244    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 19 10:34:43.108428 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108258    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-system-cni-dir\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.108428 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108331    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-system-cni-dir\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.108428 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108362    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-cni-binary-copy\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.108428 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108399    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-kubelet\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.108428 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-run-systemd\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.108756 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108432    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/630f71c6-b3e9-4935-8c99-921f92d7494c-dbus\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:43.108756 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108449    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-modprobe-d\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.108756 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108472    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znzgs\" (UniqueName: \"kubernetes.io/projected/fd50bc98-a748-4587-aacf-56daf296fe88-kube-api-access-znzgs\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.108756 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108546    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-sys-fs\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.108756 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108579    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-sysctl-conf\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.108756 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108606    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw6vr\" (UniqueName: \"kubernetes.io/projected/5492514c-c9bf-4516-9314-d1adda92b9ff-kube-api-access-xw6vr\") pod \"node-resolver-wr84k\" (UID: \"5492514c-c9bf-4516-9314-d1adda92b9ff\") " pod="openshift-dns/node-resolver-wr84k"
Sep 19 10:34:43.108756 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2c21f7fa-a5d2-4c47-9e12-7ee4a905e870-serviceca\") pod \"node-ca-qrxfh\" (UID: \"2c21f7fa-a5d2-4c47-9e12-7ee4a905e870\") " pod="openshift-image-registry/node-ca-qrxfh"
Sep 19 10:34:43.108756 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108658    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-slash\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.108756 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108693    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd50bc98-a748-4587-aacf-56daf296fe88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108764    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-multus-daemon-config\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108790    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-node-log\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108813    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-run\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd50bc98-a748-4587-aacf-56daf296fe88-cni-binary-copy\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5492514c-c9bf-4516-9314-d1adda92b9ff-hosts-file\") pod \"node-resolver-wr84k\" (UID: \"5492514c-c9bf-4516-9314-d1adda92b9ff\") " pod="openshift-dns/node-resolver-wr84k"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108880    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108905    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-registration-dir\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108927    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-run-openvswitch\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108953    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878zs\" (UniqueName: \"kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs\") pod \"network-check-target-9f2kq\" (UID: \"4c9f2fc8-10e3-4bda-8b12-8f05918712f2\") " pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108978    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-var-lib-cni-bin\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109008    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs\") pod \"network-metrics-daemon-lnqqz\" (UID: \"f85be02b-a097-4f90-844c-21d54e7dc53d\") " pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109034    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-device-dir\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109056    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-etc-selinux\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-lib-modules\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109101    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f82g5\" (UniqueName: \"kubernetes.io/projected/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-kube-api-access-f82g5\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.109177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109126    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-run-multus-certs\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4730dd7-e761-4497-bf79-2b72d0828b8f-host-slash\") pod \"iptables-alerter-xchfp\" (UID: \"e4730dd7-e761-4497-bf79-2b72d0828b8f\") " pod="openshift-network-operator/iptables-alerter-xchfp"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109163    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2c21f7fa-a5d2-4c47-9e12-7ee4a905e870-serviceca\") pod \"node-ca-qrxfh\" (UID: \"2c21f7fa-a5d2-4c47-9e12-7ee4a905e870\") " pod="openshift-image-registry/node-ca-qrxfh"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109171    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-cni-netd\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109192    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-sys\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.108986    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-cni-binary-copy\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109221    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-cnibin\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109245    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-multus-daemon-config\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109284    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-cnibin\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109324    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-var-lib-cni-multus\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.109366    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109430    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-var-lib-cni-bin\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109432    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-socket-dir\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109471    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-run-multus-certs\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.109500    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs podName:f85be02b-a097-4f90-844c-21d54e7dc53d nodeName:}" failed. No retries permitted until 2026-09-19 10:34:43.6094806 +0000 UTC m=+4.521174114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs") pod "network-metrics-daemon-lnqqz" (UID: "f85be02b-a097-4f90-844c-21d54e7dc53d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109510    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-host-var-lib-cni-multus\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109543    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-var-lib-kubelet\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.109887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109589    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/347ba7f0-42f9-43c2-b54f-55466a818064-konnectivity-ca\") pod \"konnectivity-agent-5xl48\" (UID: \"347ba7f0-42f9-43c2-b54f-55466a818064\") " pod="kube-system/konnectivity-agent-5xl48"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109614    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lt8w\" (UniqueName: \"kubernetes.io/projected/ba18fe0c-ca67-4702-8160-e4490b615f24-kube-api-access-8lt8w\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109660    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd50bc98-a748-4587-aacf-56daf296fe88-system-cni-dir\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-etc-kubernetes\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-run-netns\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109787    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-etc-kubernetes\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109802    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109833    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ba18fe0c-ca67-4702-8160-e4490b615f24-ovnkube-config\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109860    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ba18fe0c-ca67-4702-8160-e4490b615f24-env-overrides\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-multus-socket-dir-parent\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109911    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-etc-openvswitch\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109942    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-log-socket\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109965    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ba18fe0c-ca67-4702-8160-e4490b615f24-ovn-node-metrics-cert\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109975    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-multus-socket-dir-parent\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.109987    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-kubernetes\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.110006    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-systemd\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.110519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.110020    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-tuned\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.111096 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.110040    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd50bc98-a748-4587-aacf-56daf296fe88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.111096 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.110071    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-multus-conf-dir\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.111096 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.110087    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8wmw\" (UniqueName: \"kubernetes.io/projected/f85be02b-a097-4f90-844c-21d54e7dc53d-kube-api-access-j8wmw\") pod \"network-metrics-daemon-lnqqz\" (UID: \"f85be02b-a097-4f90-844c-21d54e7dc53d\") " pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:43.111096 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.110101    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfjrb\" (UniqueName: \"kubernetes.io/projected/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-kube-api-access-wfjrb\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.111096 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.110154    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/347ba7f0-42f9-43c2-b54f-55466a818064-konnectivity-ca\") pod \"konnectivity-agent-5xl48\" (UID: \"347ba7f0-42f9-43c2-b54f-55466a818064\") " pod="kube-system/konnectivity-agent-5xl48"
Sep 19 10:34:43.111096 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.110156    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-multus-conf-dir\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.112808 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.112786    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/347ba7f0-42f9-43c2-b54f-55466a818064-agent-certs\") pod \"konnectivity-agent-5xl48\" (UID: \"347ba7f0-42f9-43c2-b54f-55466a818064\") " pod="kube-system/konnectivity-agent-5xl48"
Sep 19 10:34:43.113595 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.113554    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal" event={"ID":"4ca6b3a31a573fb44477ffc56660c8ae","Type":"ContainerStarted","Data":"ce8f6fa25b9962b5b519009da8e6644af2c8204c11a7a7b6ea6a7de46ee48ece"}
Sep 19 10:34:43.114812 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.114716    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal" event={"ID":"b8679eb2ddf3de7904fb5919a04512a3","Type":"ContainerStarted","Data":"0b92c3f24313817b2ba1fde9281331ca4adc05f6139aa484b6a0d8eb79721484"}
Sep 19 10:34:43.116472 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.116453    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzrb5\" (UniqueName: \"kubernetes.io/projected/2c21f7fa-a5d2-4c47-9e12-7ee4a905e870-kube-api-access-tzrb5\") pod \"node-ca-qrxfh\" (UID: \"2c21f7fa-a5d2-4c47-9e12-7ee4a905e870\") " pod="openshift-image-registry/node-ca-qrxfh"
Sep 19 10:34:43.116821 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.116805    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv6b2\" (UniqueName: \"kubernetes.io/projected/44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae-kube-api-access-fv6b2\") pod \"multus-8vvd2\" (UID: \"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae\") " pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.118842 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.118826    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 10:34:43.118842 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.118845    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 10:34:43.118999 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.118857    2579 projected.go:196] Error preparing data for projected volume kube-api-access-878zs for pod openshift-network-diagnostics/network-check-target-9f2kq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:34:43.118999 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.118920    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs podName:4c9f2fc8-10e3-4bda-8b12-8f05918712f2 nodeName:}" failed. No retries permitted until 2026-09-19 10:34:43.618907434 +0000 UTC m=+4.530600951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-878zs" (UniqueName: "kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs") pod "network-check-target-9f2kq" (UID: "4c9f2fc8-10e3-4bda-8b12-8f05918712f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:34:43.122949 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.122927    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8wmw\" (UniqueName: \"kubernetes.io/projected/f85be02b-a097-4f90-844c-21d54e7dc53d-kube-api-access-j8wmw\") pod \"network-metrics-daemon-lnqqz\" (UID: \"f85be02b-a097-4f90-844c-21d54e7dc53d\") " pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:43.210675 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210648    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4730dd7-e761-4497-bf79-2b72d0828b8f-host-slash\") pod \"iptables-alerter-xchfp\" (UID: \"e4730dd7-e761-4497-bf79-2b72d0828b8f\") " pod="openshift-network-operator/iptables-alerter-xchfp"
Sep 19 10:34:43.210826 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210680    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-cni-netd\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.210826 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210737    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-sys\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.210826 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210761    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-socket-dir\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.210826 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210771    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e4730dd7-e761-4497-bf79-2b72d0828b8f-host-slash\") pod \"iptables-alerter-xchfp\" (UID: \"e4730dd7-e761-4497-bf79-2b72d0828b8f\") " pod="openshift-network-operator/iptables-alerter-xchfp"
Sep 19 10:34:43.210826 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-var-lib-kubelet\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.210826 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lt8w\" (UniqueName: \"kubernetes.io/projected/ba18fe0c-ca67-4702-8160-e4490b615f24-kube-api-access-8lt8w\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210828    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd50bc98-a748-4587-aacf-56daf296fe88-system-cni-dir\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210843    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-sys\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210850    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-run-netns\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210861    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-var-lib-kubelet\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210876    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210912    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-cni-netd\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210909    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd50bc98-a748-4587-aacf-56daf296fe88-system-cni-dir\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210926    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210928    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-socket-dir\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210924    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-run-netns\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ba18fe0c-ca67-4702-8160-e4490b615f24-ovnkube-config\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.210988    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ba18fe0c-ca67-4702-8160-e4490b615f24-env-overrides\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211009    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-etc-openvswitch\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211031    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-log-socket\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211054    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ba18fe0c-ca67-4702-8160-e4490b615f24-ovn-node-metrics-cert\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-kubernetes\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.211085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211078    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-etc-openvswitch\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211098    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-systemd\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211118    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-tuned\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211125    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-log-socket\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211137    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd50bc98-a748-4587-aacf-56daf296fe88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211182    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfjrb\" (UniqueName: \"kubernetes.io/projected/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-kube-api-access-wfjrb\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211185    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-systemd\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211218    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e4730dd7-e761-4497-bf79-2b72d0828b8f-iptables-alerter-script\") pod \"iptables-alerter-xchfp\" (UID: \"e4730dd7-e761-4497-bf79-2b72d0828b8f\") " pod="openshift-network-operator/iptables-alerter-xchfp"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211245    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-var-lib-openvswitch\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-cni-bin\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211291    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211313    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-host\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211335    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-tmp\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211360    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-systemd-units\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211382    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-run-ovn\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-run-ovn-kubernetes\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-var-lib-openvswitch\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.211785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211429    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd50bc98-a748-4587-aacf-56daf296fe88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211453    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5492514c-c9bf-4516-9314-d1adda92b9ff-tmp-dir\") pod \"node-resolver-wr84k\" (UID: \"5492514c-c9bf-4516-9314-d1adda92b9ff\") " pod="openshift-dns/node-resolver-wr84k"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211480    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86c4v\" (UniqueName: \"kubernetes.io/projected/e4730dd7-e761-4497-bf79-2b72d0828b8f-kube-api-access-86c4v\") pod \"iptables-alerter-xchfp\" (UID: \"e4730dd7-e761-4497-bf79-2b72d0828b8f\") " pod="openshift-network-operator/iptables-alerter-xchfp"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211485    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-kubernetes\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211487    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ba18fe0c-ca67-4702-8160-e4490b615f24-env-overrides\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-sysconfig\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211547    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-sysctl-d\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211630    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-sysconfig\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211645    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ba18fe0c-ca67-4702-8160-e4490b615f24-ovnkube-config\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211810    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-sysctl-d\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211816    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd50bc98-a748-4587-aacf-56daf296fe88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211856    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-run-ovn\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211874    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5492514c-c9bf-4516-9314-d1adda92b9ff-tmp-dir\") pod \"node-resolver-wr84k\" (UID: \"5492514c-c9bf-4516-9314-d1adda92b9ff\") " pod="openshift-dns/node-resolver-wr84k"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211897    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-host\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.211905    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211936    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-systemd-units\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211945    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ba18fe0c-ca67-4702-8160-e4490b615f24-ovnkube-script-lib\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.212657 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.211970    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret podName:630f71c6-b3e9-4935-8c99-921f92d7494c nodeName:}" failed. No retries permitted until 2026-09-19 10:34:43.711949303 +0000 UTC m=+4.623642819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret") pod "global-pull-secret-syncer-vr6wb" (UID: "630f71c6-b3e9-4935-8c99-921f92d7494c") : object "kube-system"/"original-pull-secret" not registered
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.211975    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-run-ovn-kubernetes\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212001    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/630f71c6-b3e9-4935-8c99-921f92d7494c-kubelet-config\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212036    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd50bc98-a748-4587-aacf-56daf296fe88-cnibin\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212104    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/630f71c6-b3e9-4935-8c99-921f92d7494c-kubelet-config\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212140    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd50bc98-a748-4587-aacf-56daf296fe88-cnibin\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212160    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e4730dd7-e761-4497-bf79-2b72d0828b8f-iptables-alerter-script\") pod \"iptables-alerter-xchfp\" (UID: \"e4730dd7-e761-4497-bf79-2b72d0828b8f\") " pod="openshift-network-operator/iptables-alerter-xchfp"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212173    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-kubelet\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212278    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-cni-bin\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212320    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-run-systemd\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212344    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/630f71c6-b3e9-4935-8c99-921f92d7494c-dbus\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212383    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-modprobe-d\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212387    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-run-systemd\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znzgs\" (UniqueName: \"kubernetes.io/projected/fd50bc98-a748-4587-aacf-56daf296fe88-kube-api-access-znzgs\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-sys-fs\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212489    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-modprobe-d\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212495    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ba18fe0c-ca67-4702-8160-e4490b615f24-ovnkube-script-lib\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.213419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-sysctl-conf\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212545    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-kubelet\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212551    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw6vr\" (UniqueName: \"kubernetes.io/projected/5492514c-c9bf-4516-9314-d1adda92b9ff-kube-api-access-xw6vr\") pod \"node-resolver-wr84k\" (UID: \"5492514c-c9bf-4516-9314-d1adda92b9ff\") " pod="openshift-dns/node-resolver-wr84k"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212601    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-slash\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd50bc98-a748-4587-aacf-56daf296fe88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212657    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-node-log\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212676    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd50bc98-a748-4587-aacf-56daf296fe88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-run\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd50bc98-a748-4587-aacf-56daf296fe88-cni-binary-copy\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212780    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5492514c-c9bf-4516-9314-d1adda92b9ff-hosts-file\") pod \"node-resolver-wr84k\" (UID: \"5492514c-c9bf-4516-9314-d1adda92b9ff\") " pod="openshift-dns/node-resolver-wr84k"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212791    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-sys-fs\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212803    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212833    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-node-log\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212833    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/630f71c6-b3e9-4935-8c99-921f92d7494c-dbus\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212891    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-registration-dir\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212899    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5492514c-c9bf-4516-9314-d1adda92b9ff-hosts-file\") pod \"node-resolver-wr84k\" (UID: \"5492514c-c9bf-4516-9314-d1adda92b9ff\") " pod="openshift-dns/node-resolver-wr84k"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212928    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.214117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212933    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-run-openvswitch\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212955    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-registration-dir\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.212975    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-host-slash\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.213029    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ba18fe0c-ca67-4702-8160-e4490b615f24-run-openvswitch\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.213076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-device-dir\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.213093    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-sysctl-conf\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.213108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-etc-selinux\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.213142    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-run\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.213145    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-lib-modules\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.213191    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-etc-selinux\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.213243    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-lib-modules\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.213247    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-device-dir\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.213279    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f82g5\" (UniqueName: \"kubernetes.io/projected/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-kube-api-access-f82g5\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.213479    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd50bc98-a748-4587-aacf-56daf296fe88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.213652    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd50bc98-a748-4587-aacf-56daf296fe88-cni-binary-copy\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.213657    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ba18fe0c-ca67-4702-8160-e4490b615f24-ovn-node-metrics-cert\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.213939    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-etc-tuned\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.214659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.214438    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-tmp\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.218423 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.218401    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lt8w\" (UniqueName: \"kubernetes.io/projected/ba18fe0c-ca67-4702-8160-e4490b615f24-kube-api-access-8lt8w\") pod \"ovnkube-node-wzspq\" (UID: \"ba18fe0c-ca67-4702-8160-e4490b615f24\") " pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.221974 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.221955    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfjrb\" (UniqueName: \"kubernetes.io/projected/6bf01c25-00ab-4e2c-bdfb-c95e488330e9-kube-api-access-wfjrb\") pod \"aws-ebs-csi-driver-node-jpmjk\" (UID: \"6bf01c25-00ab-4e2c-bdfb-c95e488330e9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.225450 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.225398    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86c4v\" (UniqueName: \"kubernetes.io/projected/e4730dd7-e761-4497-bf79-2b72d0828b8f-kube-api-access-86c4v\") pod \"iptables-alerter-xchfp\" (UID: \"e4730dd7-e761-4497-bf79-2b72d0828b8f\") " pod="openshift-network-operator/iptables-alerter-xchfp"
Sep 19 10:34:43.227985 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.227930    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw6vr\" (UniqueName: \"kubernetes.io/projected/5492514c-c9bf-4516-9314-d1adda92b9ff-kube-api-access-xw6vr\") pod \"node-resolver-wr84k\" (UID: \"5492514c-c9bf-4516-9314-d1adda92b9ff\") " pod="openshift-dns/node-resolver-wr84k"
Sep 19 10:34:43.228403 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.228384    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f82g5\" (UniqueName: \"kubernetes.io/projected/4f8c8161-e9d3-4086-879b-82c7a1c4b6c8-kube-api-access-f82g5\") pod \"tuned-cl9lw\" (UID: \"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8\") " pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.229650 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.229629    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znzgs\" (UniqueName: \"kubernetes.io/projected/fd50bc98-a748-4587-aacf-56daf296fe88-kube-api-access-znzgs\") pod \"multus-additional-cni-plugins-p4shs\" (UID: \"fd50bc98-a748-4587-aacf-56daf296fe88\") " pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.307856 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.307812    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qrxfh"
Sep 19 10:34:43.315393 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.315370    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5xl48"
Sep 19 10:34:43.323875 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.323852    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8vvd2"
Sep 19 10:34:43.334420 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.334403    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cl9lw"
Sep 19 10:34:43.340159 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.340137    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p4shs"
Sep 19 10:34:43.346838 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.346818    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk"
Sep 19 10:34:43.353204 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.353183    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wr84k"
Sep 19 10:34:43.359742 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.359725    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xchfp"
Sep 19 10:34:43.364514 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.364490    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:34:43.422137 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.422118    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 10:34:43.616645 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.616555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs\") pod \"network-metrics-daemon-lnqqz\" (UID: \"f85be02b-a097-4f90-844c-21d54e7dc53d\") " pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:43.616795 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.616709    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:34:43.616795 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.616779    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs podName:f85be02b-a097-4f90-844c-21d54e7dc53d nodeName:}" failed. No retries permitted until 2026-09-19 10:34:44.616760679 +0000 UTC m=+5.528454210 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs") pod "network-metrics-daemon-lnqqz" (UID: "f85be02b-a097-4f90-844c-21d54e7dc53d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:34:43.717297 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.717259    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878zs\" (UniqueName: \"kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs\") pod \"network-check-target-9f2kq\" (UID: \"4c9f2fc8-10e3-4bda-8b12-8f05918712f2\") " pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:43.717474 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:43.717318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:43.717474 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.717403    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 10:34:43.717474 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.717408    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 10:34:43.717474 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.717432    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 10:34:43.717474 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.717444    2579 projected.go:196] Error preparing data for projected volume kube-api-access-878zs for pod openshift-network-diagnostics/network-check-target-9f2kq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:34:43.717474 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.717453    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret podName:630f71c6-b3e9-4935-8c99-921f92d7494c nodeName:}" failed. No retries permitted until 2026-09-19 10:34:44.717440668 +0000 UTC m=+5.629134166 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret") pod "global-pull-secret-syncer-vr6wb" (UID: "630f71c6-b3e9-4935-8c99-921f92d7494c") : object "kube-system"/"original-pull-secret" not registered
Sep 19 10:34:43.717719 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:43.717496    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs podName:4c9f2fc8-10e3-4bda-8b12-8f05918712f2 nodeName:}" failed. No retries permitted until 2026-09-19 10:34:44.717481059 +0000 UTC m=+5.629174562 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-878zs" (UniqueName: "kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs") pod "network-check-target-9f2kq" (UID: "4c9f2fc8-10e3-4bda-8b12-8f05918712f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:34:44.006860 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:44.006818    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c21f7fa_a5d2_4c47_9e12_7ee4a905e870.slice/crio-d0157b013c996d0618221f5eb348417e5ebf87f444d4b77dd8bccb2b0ddaa2a0 WatchSource:0}: Error finding container d0157b013c996d0618221f5eb348417e5ebf87f444d4b77dd8bccb2b0ddaa2a0: Status 404 returned error can't find the container with id d0157b013c996d0618221f5eb348417e5ebf87f444d4b77dd8bccb2b0ddaa2a0
Sep 19 10:34:44.007592 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:44.007567    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba18fe0c_ca67_4702_8160_e4490b615f24.slice/crio-f5d58eb47e26a76e627f154917adee25612f61ca7934d95794293b7cdea28f75 WatchSource:0}: Error finding container f5d58eb47e26a76e627f154917adee25612f61ca7934d95794293b7cdea28f75: Status 404 returned error can't find the container with id f5d58eb47e26a76e627f154917adee25612f61ca7934d95794293b7cdea28f75
Sep 19 10:34:44.009966 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:34:44.009378    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bf01c25_00ab_4e2c_bdfb_c95e488330e9.slice/crio-3ec339a68febc162bda78b9aba8c4fc4cbdd213338b3d0cb21ff6a7ab3df2318 WatchSource:0}: Error finding container 3ec339a68febc162bda78b9aba8c4fc4cbdd213338b3d0cb21ff6a7ab3df2318: Status 404 returned error can't find the container with id 3ec339a68febc162bda78b9aba8c4fc4cbdd213338b3d0cb21ff6a7ab3df2318
Sep 19 10:34:44.066907 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.066881    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 10:29:43 +0000 UTC" deadline="2028-04-24 14:57:12.610818873 +0000 UTC"
Sep 19 10:34:44.066907 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.066907    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13996h22m28.543915642s"
Sep 19 10:34:44.109920 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.109898    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:44.110006 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:44.109987    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:34:44.117676 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.117641    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xchfp" event={"ID":"e4730dd7-e761-4497-bf79-2b72d0828b8f","Type":"ContainerStarted","Data":"74a00a3983f0e9261b18b6e9b87f91e17d1a9a9c2edd95b609d47fc89ced2a46"}
Sep 19 10:34:44.118990 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.118967    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cl9lw" event={"ID":"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8","Type":"ContainerStarted","Data":"a88e00be650590285de06d394727754f6803f8c0e0024e48dbce766fdca1fc0d"}
Sep 19 10:34:44.120213 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.120189    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8vvd2" event={"ID":"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae","Type":"ContainerStarted","Data":"ff2ed2722f8b585e1a5e5c184703e0357e6ab8a59945cc8c28edc279ae974d5d"}
Sep 19 10:34:44.121474 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.121454    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5xl48" event={"ID":"347ba7f0-42f9-43c2-b54f-55466a818064","Type":"ContainerStarted","Data":"ce4743bce9fefaf8b4936377d00afe833f4fd1923a8c554eed5bca2087644e78"}
Sep 19 10:34:44.122870 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.122851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk" event={"ID":"6bf01c25-00ab-4e2c-bdfb-c95e488330e9","Type":"ContainerStarted","Data":"3ec339a68febc162bda78b9aba8c4fc4cbdd213338b3d0cb21ff6a7ab3df2318"}
Sep 19 10:34:44.124866 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.124844    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal" event={"ID":"b8679eb2ddf3de7904fb5919a04512a3","Type":"ContainerStarted","Data":"13b805fa459e3d90065330a7584746eb241819119bf28979633d6b600e2c23dd"}
Sep 19 10:34:44.126132 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.126101    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p4shs" event={"ID":"fd50bc98-a748-4587-aacf-56daf296fe88","Type":"ContainerStarted","Data":"e7b69e939dafef4fbf58f9efd6a4eddacc852d408b9665b7da9766259c64b580"}
Sep 19 10:34:44.127477 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.127454    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wr84k" event={"ID":"5492514c-c9bf-4516-9314-d1adda92b9ff","Type":"ContainerStarted","Data":"d3b9660b0e6fd014b0b7c0a3592e66973448571fe499a5860bf0064ad06fbcfd"}
Sep 19 10:34:44.128718 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.128694    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq" event={"ID":"ba18fe0c-ca67-4702-8160-e4490b615f24","Type":"ContainerStarted","Data":"f5d58eb47e26a76e627f154917adee25612f61ca7934d95794293b7cdea28f75"}
Sep 19 10:34:44.129897 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.129877    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qrxfh" event={"ID":"2c21f7fa-a5d2-4c47-9e12-7ee4a905e870","Type":"ContainerStarted","Data":"d0157b013c996d0618221f5eb348417e5ebf87f444d4b77dd8bccb2b0ddaa2a0"}
Sep 19 10:34:44.162509 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.162471    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-105.ec2.internal" podStartSLOduration=2.162439897 podStartE2EDuration="2.162439897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:34:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:34:44.162182404 +0000 UTC m=+5.073875924" watchObservedRunningTime="2026-09-19 10:34:44.162439897 +0000 UTC m=+5.074133417"
Sep 19 10:34:44.624058 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.623770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs\") pod \"network-metrics-daemon-lnqqz\" (UID: \"f85be02b-a097-4f90-844c-21d54e7dc53d\") " pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:44.624243 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:44.624185    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:34:44.624332 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:44.624251    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs podName:f85be02b-a097-4f90-844c-21d54e7dc53d nodeName:}" failed. No retries permitted until 2026-09-19 10:34:46.624230409 +0000 UTC m=+7.535923924 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs") pod "network-metrics-daemon-lnqqz" (UID: "f85be02b-a097-4f90-844c-21d54e7dc53d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:34:44.725009 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.724973    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:44.725187 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:44.725041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878zs\" (UniqueName: \"kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs\") pod \"network-check-target-9f2kq\" (UID: \"4c9f2fc8-10e3-4bda-8b12-8f05918712f2\") " pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:44.725249 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:44.725198    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 10:34:44.725249 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:44.725215    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 10:34:44.725249 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:44.725226    2579 projected.go:196] Error preparing data for projected volume kube-api-access-878zs for pod openshift-network-diagnostics/network-check-target-9f2kq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:34:44.725393 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:44.725284    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs podName:4c9f2fc8-10e3-4bda-8b12-8f05918712f2 nodeName:}" failed. No retries permitted until 2026-09-19 10:34:46.725263804 +0000 UTC m=+7.636957318 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-878zs" (UniqueName: "kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs") pod "network-check-target-9f2kq" (UID: "4c9f2fc8-10e3-4bda-8b12-8f05918712f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:34:44.725691 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:44.725669    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 10:34:44.725774 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:44.725727    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret podName:630f71c6-b3e9-4935-8c99-921f92d7494c nodeName:}" failed. No retries permitted until 2026-09-19 10:34:46.725711599 +0000 UTC m=+7.637405114 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret") pod "global-pull-secret-syncer-vr6wb" (UID: "630f71c6-b3e9-4935-8c99-921f92d7494c") : object "kube-system"/"original-pull-secret" not registered
Sep 19 10:34:45.067447 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:45.067304    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 10:29:43 +0000 UTC" deadline="2028-07-01 15:33:17.895886696 +0000 UTC"
Sep 19 10:34:45.067447 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:45.067369    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15628h58m32.828521704s"
Sep 19 10:34:45.109811 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:45.109317    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:45.109811 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:45.109456    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:34:45.110039 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:45.110023    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:45.110163 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:45.110125    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:34:45.143237 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:45.143207    2579 generic.go:356] "Generic (PLEG): container finished" podID="4ca6b3a31a573fb44477ffc56660c8ae" containerID="233468aaa521b523aaf5bc9b6b65d7130cf5287ad971df6b149ed0ebdf546e93" exitCode=0
Sep 19 10:34:45.144071 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:45.144049    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal" event={"ID":"4ca6b3a31a573fb44477ffc56660c8ae","Type":"ContainerDied","Data":"233468aaa521b523aaf5bc9b6b65d7130cf5287ad971df6b149ed0ebdf546e93"}
Sep 19 10:34:46.111303 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:46.111270    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:46.111760 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:46.111399    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:34:46.155597 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:46.155057    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal" event={"ID":"4ca6b3a31a573fb44477ffc56660c8ae","Type":"ContainerStarted","Data":"aa6566b150765a212bed04a49060d742f1fedd79ce12308d53d7b311b6bd7316"}
Sep 19 10:34:46.641061 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:46.641025    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs\") pod \"network-metrics-daemon-lnqqz\" (UID: \"f85be02b-a097-4f90-844c-21d54e7dc53d\") " pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:46.641252 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:46.641171    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:34:46.641252 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:46.641239    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs podName:f85be02b-a097-4f90-844c-21d54e7dc53d nodeName:}" failed. No retries permitted until 2026-09-19 10:34:50.641220409 +0000 UTC m=+11.552913932 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs") pod "network-metrics-daemon-lnqqz" (UID: "f85be02b-a097-4f90-844c-21d54e7dc53d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:34:46.742408 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:46.742369    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:46.742575 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:46.742438    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878zs\" (UniqueName: \"kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs\") pod \"network-check-target-9f2kq\" (UID: \"4c9f2fc8-10e3-4bda-8b12-8f05918712f2\") " pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:46.742634 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:46.742612    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 10:34:46.742634 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:46.742630    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 10:34:46.742746 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:46.742645    2579 projected.go:196] Error preparing data for projected volume kube-api-access-878zs for pod openshift-network-diagnostics/network-check-target-9f2kq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:34:46.742746 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:46.742701    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs podName:4c9f2fc8-10e3-4bda-8b12-8f05918712f2 nodeName:}" failed. No retries permitted until 2026-09-19 10:34:50.742682346 +0000 UTC m=+11.654375845 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-878zs" (UniqueName: "kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs") pod "network-check-target-9f2kq" (UID: "4c9f2fc8-10e3-4bda-8b12-8f05918712f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:34:46.742849 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:46.742771    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 10:34:46.742849 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:46.742806    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret podName:630f71c6-b3e9-4935-8c99-921f92d7494c nodeName:}" failed. No retries permitted until 2026-09-19 10:34:50.742794857 +0000 UTC m=+11.654488359 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret") pod "global-pull-secret-syncer-vr6wb" (UID: "630f71c6-b3e9-4935-8c99-921f92d7494c") : object "kube-system"/"original-pull-secret" not registered
Sep 19 10:34:47.109804 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:47.109172    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:47.109804 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:47.109222    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:47.109804 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:47.109313    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:34:47.109804 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:47.109403    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:34:48.110041 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:48.110006    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:48.110510 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:48.110171    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:34:49.109496 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:49.109461    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:49.109686 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:49.109461    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:49.109686 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:49.109632    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:34:49.109866 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:49.109789    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:34:50.109552 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:50.109406    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:50.109552 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:50.109545    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:34:50.678071 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:50.678034    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs\") pod \"network-metrics-daemon-lnqqz\" (UID: \"f85be02b-a097-4f90-844c-21d54e7dc53d\") " pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:50.678256 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:50.678198    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:34:50.678316 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:50.678257    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs podName:f85be02b-a097-4f90-844c-21d54e7dc53d nodeName:}" failed. No retries permitted until 2026-09-19 10:34:58.6782376 +0000 UTC m=+19.589931104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs") pod "network-metrics-daemon-lnqqz" (UID: "f85be02b-a097-4f90-844c-21d54e7dc53d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:34:50.779061 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:50.779031    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:50.779260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:50.779084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878zs\" (UniqueName: \"kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs\") pod \"network-check-target-9f2kq\" (UID: \"4c9f2fc8-10e3-4bda-8b12-8f05918712f2\") " pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:50.779260 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:50.779201    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 10:34:50.779260 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:50.779220    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 10:34:50.779260 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:50.779230    2579 projected.go:196] Error preparing data for projected volume kube-api-access-878zs for pod openshift-network-diagnostics/network-check-target-9f2kq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:34:50.779510 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:50.779271    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs podName:4c9f2fc8-10e3-4bda-8b12-8f05918712f2 nodeName:}" failed. No retries permitted until 2026-09-19 10:34:58.779258998 +0000 UTC m=+19.690952495 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-878zs" (UniqueName: "kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs") pod "network-check-target-9f2kq" (UID: "4c9f2fc8-10e3-4bda-8b12-8f05918712f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:34:50.779689 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:50.779586    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 10:34:50.779689 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:50.779632    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret podName:630f71c6-b3e9-4935-8c99-921f92d7494c nodeName:}" failed. No retries permitted until 2026-09-19 10:34:58.779620483 +0000 UTC m=+19.691313980 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret") pod "global-pull-secret-syncer-vr6wb" (UID: "630f71c6-b3e9-4935-8c99-921f92d7494c") : object "kube-system"/"original-pull-secret" not registered
Sep 19 10:34:51.110697 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:51.110194    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:51.110697 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:51.110313    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:34:51.110697 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:51.110567    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:51.110697 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:51.110685    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:34:52.109267 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:52.109149    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:52.109467 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:52.109270    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:34:53.109741 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:53.109713    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:53.110226 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:53.109715    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:53.110226 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:53.109816    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:34:53.110226 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:53.109931    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:34:54.109427 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:54.109395    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:54.109615 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:54.109514    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:34:55.109336 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:55.109302    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:55.109336 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:55.109302    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:55.109958 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:55.109507    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:34:55.109958 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:55.109891    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:34:56.109561 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:56.109514    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:56.109958 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:56.109654    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:34:57.109492 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:57.109457    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:57.109685 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:57.109637    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:34:57.110003 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:57.109686    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:57.110003 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:57.109779    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:34:58.109464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:58.109430    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:58.109681 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:58.109554    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:34:58.737478 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:58.737445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs\") pod \"network-metrics-daemon-lnqqz\" (UID: \"f85be02b-a097-4f90-844c-21d54e7dc53d\") " pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:58.737886 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:58.737607    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:34:58.737886 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:58.737673    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs podName:f85be02b-a097-4f90-844c-21d54e7dc53d nodeName:}" failed. No retries permitted until 2026-09-19 10:35:14.737654787 +0000 UTC m=+35.649348284 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs") pod "network-metrics-daemon-lnqqz" (UID: "f85be02b-a097-4f90-844c-21d54e7dc53d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:34:58.838643 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:58.838612    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:58.838799 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:58.838674    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878zs\" (UniqueName: \"kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs\") pod \"network-check-target-9f2kq\" (UID: \"4c9f2fc8-10e3-4bda-8b12-8f05918712f2\") " pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:34:58.838799 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:58.838749    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 10:34:58.838900 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:58.838809    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 10:34:58.838900 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:58.838827    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret podName:630f71c6-b3e9-4935-8c99-921f92d7494c nodeName:}" failed. No retries permitted until 2026-09-19 10:35:14.838806039 +0000 UTC m=+35.750499539 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret") pod "global-pull-secret-syncer-vr6wb" (UID: "630f71c6-b3e9-4935-8c99-921f92d7494c") : object "kube-system"/"original-pull-secret" not registered
Sep 19 10:34:58.838900 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:58.838828    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 10:34:58.838900 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:58.838849    2579 projected.go:196] Error preparing data for projected volume kube-api-access-878zs for pod openshift-network-diagnostics/network-check-target-9f2kq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:34:58.838900 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:58.838884    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs podName:4c9f2fc8-10e3-4bda-8b12-8f05918712f2 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:14.838873277 +0000 UTC m=+35.750566775 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-878zs" (UniqueName: "kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs") pod "network-check-target-9f2kq" (UID: "4c9f2fc8-10e3-4bda-8b12-8f05918712f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:34:59.109754 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:59.109682    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:34:59.109907 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:34:59.109682    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:34:59.109907 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:59.109828    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:34:59.109907 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:34:59.109884    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:35:00.109384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:00.109344    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:00.109867 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:00.109471    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:35:01.110777 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:01.110749    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:35:01.111134 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:01.110868    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:35:01.111134 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:01.110914    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:35:01.111134 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:01.110999    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:35:02.109674 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.109462    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:02.109803 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:02.109779    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:35:02.193180 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.193148    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8vvd2" event={"ID":"44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae","Type":"ContainerStarted","Data":"f64d86c78383a4ca0757419a255e1b0eb05f6068f3a1e69a604c6a2a9e77f627"}
Sep 19 10:35:02.194654 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.194627    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5xl48" event={"ID":"347ba7f0-42f9-43c2-b54f-55466a818064","Type":"ContainerStarted","Data":"d6ce47fe89e47e9e1c71fcb57b50c4a56d9cd170c0f891ff23eb8cca853473ff"}
Sep 19 10:35:02.195979 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.195956    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk" event={"ID":"6bf01c25-00ab-4e2c-bdfb-c95e488330e9","Type":"ContainerStarted","Data":"7d4814e0a77424fe5c5fbfd4c7b273131c885bacabea697f73b100935613ab69"}
Sep 19 10:35:02.197439 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.197415    2579 generic.go:356] "Generic (PLEG): container finished" podID="fd50bc98-a748-4587-aacf-56daf296fe88" containerID="d594db62b475f25c73f697e248a13c0086f3cacea7e9bfefeab857d573119e57" exitCode=0
Sep 19 10:35:02.197538 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.197482    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p4shs" event={"ID":"fd50bc98-a748-4587-aacf-56daf296fe88","Type":"ContainerDied","Data":"d594db62b475f25c73f697e248a13c0086f3cacea7e9bfefeab857d573119e57"}
Sep 19 10:35:02.198981 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.198953    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wr84k" event={"ID":"5492514c-c9bf-4516-9314-d1adda92b9ff","Type":"ContainerStarted","Data":"a1187c37c0531220a96c468732c8206ef835243944fc87db1aaf4384f4492d8d"}
Sep 19 10:35:02.201808 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.201792    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:35:02.202195 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.202176    2579 generic.go:356] "Generic (PLEG): container finished" podID="ba18fe0c-ca67-4702-8160-e4490b615f24" containerID="18cf909c26d7d689e970244993943d60d0dad9e264cc34acd6c08b91b78eaec7" exitCode=1
Sep 19 10:35:02.202258 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.202240    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq" event={"ID":"ba18fe0c-ca67-4702-8160-e4490b615f24","Type":"ContainerStarted","Data":"a01514c691e566fcba5503feedc2101da65dad60ca88008debeb41abd38d85b2"}
Sep 19 10:35:02.202327 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.202313    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq" event={"ID":"ba18fe0c-ca67-4702-8160-e4490b615f24","Type":"ContainerStarted","Data":"36f4995133eddd47a7947ae8d84c12929a576886595fd565d434155c0579035f"}
Sep 19 10:35:02.202369 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.202351    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq" event={"ID":"ba18fe0c-ca67-4702-8160-e4490b615f24","Type":"ContainerStarted","Data":"125ab93017d8b8aa3916cc1fec67e3df87b75b274b9a1b12f22cd5d3339b1b83"}
Sep 19 10:35:02.202415 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.202380    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq" event={"ID":"ba18fe0c-ca67-4702-8160-e4490b615f24","Type":"ContainerDied","Data":"18cf909c26d7d689e970244993943d60d0dad9e264cc34acd6c08b91b78eaec7"}
Sep 19 10:35:02.202415 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.202408    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq" event={"ID":"ba18fe0c-ca67-4702-8160-e4490b615f24","Type":"ContainerStarted","Data":"f9e6911017671ddc1600c8e484ee2ff3524ba408c23268fe4dfe192b458286eb"}
Sep 19 10:35:02.203516 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.203495    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qrxfh" event={"ID":"2c21f7fa-a5d2-4c47-9e12-7ee4a905e870","Type":"ContainerStarted","Data":"1788eaa95c3374e2d8ed07a48648c72d365549f7277d09a3d55de34715f7c675"}
Sep 19 10:35:02.204793 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.204777    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cl9lw" event={"ID":"4f8c8161-e9d3-4086-879b-82c7a1c4b6c8","Type":"ContainerStarted","Data":"cf2d72050a8ded413f8e1ccf69c169b5d7d4d1a960ed0600e7dd3f1b60f8b28b"}
Sep 19 10:35:02.215813 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.213110    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8vvd2" podStartSLOduration=3.833962156 podStartE2EDuration="21.213094415s" totalImagesPullingTime="17.379132259s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:02.212580458 +0000 UTC m=+23.124273976" watchObservedRunningTime="2026-09-19 10:35:02.213094415 +0000 UTC m=+23.124787936"
Sep 19 10:35:02.215813 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.213611    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-105.ec2.internal" podStartSLOduration=20.213595076 podStartE2EDuration="20.213595076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:34:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:34:46.1746597 +0000 UTC m=+7.086353220" watchObservedRunningTime="2026-09-19 10:35:02.213595076 +0000 UTC m=+23.125288596"
Sep 19 10:35:02.245594 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.245546    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cl9lw" podStartSLOduration=3.868031441 podStartE2EDuration="21.245509909s" totalImagesPullingTime="17.377478468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:02.245208626 +0000 UTC m=+23.156902180" watchObservedRunningTime="2026-09-19 10:35:02.245509909 +0000 UTC m=+23.157203427"
Sep 19 10:35:02.245741 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.245665    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qrxfh" podStartSLOduration=4.091465487 podStartE2EDuration="21.24565817s" totalImagesPullingTime="17.154192683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:02.227868059 +0000 UTC m=+23.139561581" watchObservedRunningTime="2026-09-19 10:35:02.24565817 +0000 UTC m=+23.157351693"
Sep 19 10:35:02.296439 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.296248    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wr84k" podStartSLOduration=3.924680749 podStartE2EDuration="21.296232775s" totalImagesPullingTime="17.371552026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:02.295837238 +0000 UTC m=+23.207530757" watchObservedRunningTime="2026-09-19 10:35:02.296232775 +0000 UTC m=+23.207926293"
Sep 19 10:35:02.671082 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:02.671057    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 19 10:35:03.081120 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:03.081020    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-19T10:35:02.67107354Z","UUID":"5610e0de-c47f-4e76-a122-f59cdbdcde1e","Handler":null,"Name":"","Endpoint":""}
Sep 19 10:35:03.082860 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:03.082835    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 19 10:35:03.082977 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:03.082869    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 19 10:35:03.109067 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:03.109043    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:35:03.109191 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:03.109079    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:35:03.109191 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:03.109176    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:35:03.109384 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:03.109358    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:35:03.209534 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:03.209478    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xchfp" event={"ID":"e4730dd7-e761-4497-bf79-2b72d0828b8f","Type":"ContainerStarted","Data":"de2221045291c6e48ff0f2af254d0efe3676ee37519f47269df08f922fff7ba0"}
Sep 19 10:35:03.211979 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:03.211951    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk" event={"ID":"6bf01c25-00ab-4e2c-bdfb-c95e488330e9","Type":"ContainerStarted","Data":"fb25aa60e341d8b1ff569a7934a15a95dfc6f0cac097b98043c17d7f72eb9d7c"}
Sep 19 10:35:03.216204 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:03.216148    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:35:03.217083 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:03.216780    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq" event={"ID":"ba18fe0c-ca67-4702-8160-e4490b615f24","Type":"ContainerStarted","Data":"535456fff60036f4175ba7e35a2daa78c046913a7943a42075190eb70218846c"}
Sep 19 10:35:03.224290 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:03.224246    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5xl48" podStartSLOduration=8.746627927 podStartE2EDuration="22.224229323s" totalImagesPullingTime="13.477601396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:02.341790767 +0000 UTC m=+23.253484286" watchObservedRunningTime="2026-09-19 10:35:03.224229323 +0000 UTC m=+24.135922844"
Sep 19 10:35:03.224915 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:03.224884    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xchfp" podStartSLOduration=5.085061075 podStartE2EDuration="22.224874884s" totalImagesPullingTime="17.139813809s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:03.224148921 +0000 UTC m=+24.135842503" watchObservedRunningTime="2026-09-19 10:35:03.224874884 +0000 UTC m=+24.136568403"
Sep 19 10:35:04.109906 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:04.109819    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:04.110114 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:04.109942    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:35:04.221261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:04.221225    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk" event={"ID":"6bf01c25-00ab-4e2c-bdfb-c95e488330e9","Type":"ContainerStarted","Data":"7cbb59c945318f601f7107e04bcb39bb3fdcdc83821af697a219493544147ce9"}
Sep 19 10:35:04.239011 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:04.238968    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jpmjk" podStartSLOduration=4.163525578 podStartE2EDuration="23.238954297s" totalImagesPullingTime="19.075428719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:34:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:04.23782571 +0000 UTC m=+25.149519228" watchObservedRunningTime="2026-09-19 10:35:04.238954297 +0000 UTC m=+25.150647816"
Sep 19 10:35:05.077336 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:05.077296    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:05.109766 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:05.109736    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:35:05.109937 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:05.109862    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:35:05.109937 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:05.109894    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:35:05.110049 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:05.109975    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:35:05.175664 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:05.175620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:05.227205 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:05.227180    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:35:05.227680 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:05.227637    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq" event={"ID":"ba18fe0c-ca67-4702-8160-e4490b615f24","Type":"ContainerStarted","Data":"f352aa95c9a04f56c6909a4bfee7cb50301b55ad7cf6b390738b6d52de32c6a8"}
Sep 19 10:35:05.275040 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:05.275003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:05.528224 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:05.528149    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5xl48"
Sep 19 10:35:05.528777 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:05.528753    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5xl48"
Sep 19 10:35:06.109148 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:06.109117    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:06.109332 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:06.109230    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:35:06.229795 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:06.229762    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5xl48"
Sep 19 10:35:06.230293 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:06.230276    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5xl48"
Sep 19 10:35:06.293559 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:06.293512    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:07.109546 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:07.109295    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:35:07.109682 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:07.109327    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:35:07.109716 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:07.109674    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:35:07.109716 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:07.109694    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:35:07.234349 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:07.234322    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:35:07.235154 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:07.234736    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq" event={"ID":"ba18fe0c-ca67-4702-8160-e4490b615f24","Type":"ContainerStarted","Data":"0f4a2e99f1da59fe79ee4d9f50a64a57dae3cd7a04352b9a5106a14747a19f35"}
Sep 19 10:35:07.235154 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:07.234951    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:35:07.235154 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:07.235116    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:35:07.235154 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:07.235145    2579 scope.go:117] "RemoveContainer" containerID="18cf909c26d7d689e970244993943d60d0dad9e264cc34acd6c08b91b78eaec7"
Sep 19 10:35:07.235338 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:07.235177    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:35:07.237125 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:07.237099    2579 generic.go:356] "Generic (PLEG): container finished" podID="fd50bc98-a748-4587-aacf-56daf296fe88" containerID="dd88ac7904bc16016d8cdd4c7390ffdf07c3ea2b3d2f1e664f3cd8f6893608d5" exitCode=0
Sep 19 10:35:07.237250 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:07.237172    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p4shs" event={"ID":"fd50bc98-a748-4587-aacf-56daf296fe88","Type":"ContainerDied","Data":"dd88ac7904bc16016d8cdd4c7390ffdf07c3ea2b3d2f1e664f3cd8f6893608d5"}
Sep 19 10:35:07.252491 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:07.252450    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:35:07.252581 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:07.252569    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:35:08.109558 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:08.109378    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:08.109687 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:08.109628    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:35:08.243742 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:08.243721    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:35:08.244139 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:08.244105    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq" event={"ID":"ba18fe0c-ca67-4702-8160-e4490b615f24","Type":"ContainerStarted","Data":"6fb418b2f63212d642204655288bc371f18a08fdffb4ce2f65b0678439957411"}
Sep 19 10:35:08.288675 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:08.288634    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq" podStartSLOduration=9.853010727000001 podStartE2EDuration="27.288618549s" totalImagesPullingTime="17.435607822s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:08.28792083 +0000 UTC m=+29.199614353" watchObservedRunningTime="2026-09-19 10:35:08.288618549 +0000 UTC m=+29.200312068"
Sep 19 10:35:08.412428 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:08.412401    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:08.515150 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:08.515095    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:08.619357 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:08.619329    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vr6wb"]
Sep 19 10:35:08.619493 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:08.619457    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:35:08.619606 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:08.619583    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:35:08.620031 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:08.620009    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9f2kq"]
Sep 19 10:35:08.620123 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:08.620110    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:08.620223 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:08.620207    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:35:08.631372 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:08.631317    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lnqqz"]
Sep 19 10:35:08.631451 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:08.631396    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:35:08.631492 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:08.631470    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:35:08.846329 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:08.846294    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:08.899714 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:08.899651    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:09.238315 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:09.238224    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:09.247989 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:09.247955    2579 generic.go:356] "Generic (PLEG): container finished" podID="fd50bc98-a748-4587-aacf-56daf296fe88" containerID="febce66f82f522b8d685626aace68d134852d12d9c88c58ec5b06578bd8aa2d7" exitCode=0
Sep 19 10:35:09.248335 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:09.248041    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p4shs" event={"ID":"fd50bc98-a748-4587-aacf-56daf296fe88","Type":"ContainerDied","Data":"febce66f82f522b8d685626aace68d134852d12d9c88c58ec5b06578bd8aa2d7"}
Sep 19 10:35:09.302392 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:09.302361    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:09.978272 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:09.978212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:10.033366 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:10.033340    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:10.109223 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:10.109197    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:10.109335 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:10.109294    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:35:10.109335 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:10.109300    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:35:10.109415 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:10.109381    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:35:10.109415 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:10.109405    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:35:10.109484 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:10.109456    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:35:10.641724 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:10.641693    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:10.742262 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:10.742235    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:11.254582 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:11.254554    2579 generic.go:356] "Generic (PLEG): container finished" podID="fd50bc98-a748-4587-aacf-56daf296fe88" containerID="c1959f6b8a96e7d5cf8095c96ad0fc1d85daaa398c8b90bc960ac20daf972700" exitCode=0
Sep 19 10:35:11.254725 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:11.254604    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p4shs" event={"ID":"fd50bc98-a748-4587-aacf-56daf296fe88","Type":"ContainerDied","Data":"c1959f6b8a96e7d5cf8095c96ad0fc1d85daaa398c8b90bc960ac20daf972700"}
Sep 19 10:35:12.108965 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:12.108935    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:35:12.109375 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:12.108935    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:35:12.109375 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:12.108935    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:12.109375 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:12.109110    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:35:12.109375 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:12.109155    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:35:12.109375 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:12.109255    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:35:13.218578 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:13.218542    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:14.110060 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.109861    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:35:14.110229 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.109925    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:35:14.110229 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:14.110145    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vr6wb" podUID="630f71c6-b3e9-4935-8c99-921f92d7494c"
Sep 19 10:35:14.110229 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.109923    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:14.110426 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:14.110226    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lnqqz" podUID="f85be02b-a097-4f90-844c-21d54e7dc53d"
Sep 19 10:35:14.110426 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:14.110277    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9f2kq" podUID="4c9f2fc8-10e3-4bda-8b12-8f05918712f2"
Sep 19 10:35:14.537222 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.537196    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-105.ec2.internal" event="NodeReady"
Sep 19 10:35:14.537625 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.537326    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 19 10:35:14.584272 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.584246    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jqhh6"]
Sep 19 10:35:14.609908 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.609880    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q7nq7"]
Sep 19 10:35:14.610227 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.610192    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:14.613669 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.613647    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 19 10:35:14.613669 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.613647    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 19 10:35:14.614002 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.613986    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-snddw\""
Sep 19 10:35:14.631488 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.631466    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jqhh6"]
Sep 19 10:35:14.631609 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.631510    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q7nq7"]
Sep 19 10:35:14.631665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.631607    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q7nq7"
Sep 19 10:35:14.634831 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.634806    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 19 10:35:14.635004 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.634954    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 19 10:35:14.635097 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.635042    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cttpt\""
Sep 19 10:35:14.635317 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.635296    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 19 10:35:14.762988 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.762913    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:14.762988 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.762953    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b45f9d0-487d-41c2-97e0-98865a07f066-cert\") pod \"ingress-canary-q7nq7\" (UID: \"0b45f9d0-487d-41c2-97e0-98865a07f066\") " pod="openshift-ingress-canary/ingress-canary-q7nq7"
Sep 19 10:35:14.762988 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.762987    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d6349d68-d16e-4a28-b8c1-40630155ed16-config-volume\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:14.763249 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.763010    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh2gn\" (UniqueName: \"kubernetes.io/projected/d6349d68-d16e-4a28-b8c1-40630155ed16-kube-api-access-rh2gn\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:14.763249 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.763030    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh7fd\" (UniqueName: \"kubernetes.io/projected/0b45f9d0-487d-41c2-97e0-98865a07f066-kube-api-access-kh7fd\") pod \"ingress-canary-q7nq7\" (UID: \"0b45f9d0-487d-41c2-97e0-98865a07f066\") " pod="openshift-ingress-canary/ingress-canary-q7nq7"
Sep 19 10:35:14.763249 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.763070    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs\") pod \"network-metrics-daemon-lnqqz\" (UID: \"f85be02b-a097-4f90-844c-21d54e7dc53d\") " pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:35:14.763249 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.763144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6349d68-d16e-4a28-b8c1-40630155ed16-tmp-dir\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:14.763249 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:14.763173    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:35:14.763249 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:14.763239    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs podName:f85be02b-a097-4f90-844c-21d54e7dc53d nodeName:}" failed. No retries permitted until 2026-09-19 10:35:46.763219002 +0000 UTC m=+67.674912500 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs") pod "network-metrics-daemon-lnqqz" (UID: "f85be02b-a097-4f90-844c-21d54e7dc53d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 10:35:14.863847 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.863812    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:35:14.864097 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.863856    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:14.864097 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.863968    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b45f9d0-487d-41c2-97e0-98865a07f066-cert\") pod \"ingress-canary-q7nq7\" (UID: \"0b45f9d0-487d-41c2-97e0-98865a07f066\") " pod="openshift-ingress-canary/ingress-canary-q7nq7"
Sep 19 10:35:14.864097 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:14.863999    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 10:35:14.864097 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.864021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d6349d68-d16e-4a28-b8c1-40630155ed16-config-volume\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:14.864097 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.864045    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh2gn\" (UniqueName: \"kubernetes.io/projected/d6349d68-d16e-4a28-b8c1-40630155ed16-kube-api-access-rh2gn\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:14.864097 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:14.864058    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 10:35:14.864097 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:14.864071    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret podName:630f71c6-b3e9-4935-8c99-921f92d7494c nodeName:}" failed. No retries permitted until 2026-09-19 10:35:46.864051096 +0000 UTC m=+67.775744592 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret") pod "global-pull-secret-syncer-vr6wb" (UID: "630f71c6-b3e9-4935-8c99-921f92d7494c") : object "kube-system"/"original-pull-secret" not registered
Sep 19 10:35:14.864097 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.864099    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh7fd\" (UniqueName: \"kubernetes.io/projected/0b45f9d0-487d-41c2-97e0-98865a07f066-kube-api-access-kh7fd\") pod \"ingress-canary-q7nq7\" (UID: \"0b45f9d0-487d-41c2-97e0-98865a07f066\") " pod="openshift-ingress-canary/ingress-canary-q7nq7"
Sep 19 10:35:14.864498 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:14.864117    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls podName:d6349d68-d16e-4a28-b8c1-40630155ed16 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:15.364099367 +0000 UTC m=+36.275792869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls") pod "dns-default-jqhh6" (UID: "d6349d68-d16e-4a28-b8c1-40630155ed16") : secret "dns-default-metrics-tls" not found
Sep 19 10:35:14.864498 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.864161    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878zs\" (UniqueName: \"kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs\") pod \"network-check-target-9f2kq\" (UID: \"4c9f2fc8-10e3-4bda-8b12-8f05918712f2\") " pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:14.864498 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.864211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6349d68-d16e-4a28-b8c1-40630155ed16-tmp-dir\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:14.864498 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:14.864299    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 10:35:14.864498 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:14.864322    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 10:35:14.864498 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:14.864336    2579 projected.go:196] Error preparing data for projected volume kube-api-access-878zs for pod openshift-network-diagnostics/network-check-target-9f2kq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:35:14.864498 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:14.864380    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs podName:4c9f2fc8-10e3-4bda-8b12-8f05918712f2 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:46.864366421 +0000 UTC m=+67.776059937 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-878zs" (UniqueName: "kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs") pod "network-check-target-9f2kq" (UID: "4c9f2fc8-10e3-4bda-8b12-8f05918712f2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 10:35:14.864848 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.864803    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6349d68-d16e-4a28-b8c1-40630155ed16-tmp-dir\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:14.864965 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.864874    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d6349d68-d16e-4a28-b8c1-40630155ed16-config-volume\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:14.868758 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.868733    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b45f9d0-487d-41c2-97e0-98865a07f066-cert\") pod \"ingress-canary-q7nq7\" (UID: \"0b45f9d0-487d-41c2-97e0-98865a07f066\") " pod="openshift-ingress-canary/ingress-canary-q7nq7"
Sep 19 10:35:14.872863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.872837    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh7fd\" (UniqueName: \"kubernetes.io/projected/0b45f9d0-487d-41c2-97e0-98865a07f066-kube-api-access-kh7fd\") pod \"ingress-canary-q7nq7\" (UID: \"0b45f9d0-487d-41c2-97e0-98865a07f066\") " pod="openshift-ingress-canary/ingress-canary-q7nq7"
Sep 19 10:35:14.872984 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.872943    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh2gn\" (UniqueName: \"kubernetes.io/projected/d6349d68-d16e-4a28-b8c1-40630155ed16-kube-api-access-rh2gn\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:14.888422 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.888394    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:14.942439 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.942413    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q7nq7"
Sep 19 10:35:14.952630 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:14.952601    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:15.367345 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.367318    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:15.367617 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.367459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:15.367617 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:15.367588    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 10:35:15.367719 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:15.367633    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls podName:d6349d68-d16e-4a28-b8c1-40630155ed16 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:16.367621778 +0000 UTC m=+37.279315274 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls") pod "dns-default-jqhh6" (UID: "d6349d68-d16e-4a28-b8c1-40630155ed16") : secret "dns-default-metrics-tls" not found
Sep 19 10:35:15.468028 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.468005    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:15.531925 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.531901    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-66fmf"]
Sep 19 10:35:15.547865 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.547847    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:15.550917 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.550895    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 19 10:35:15.551271 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.550904    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 19 10:35:15.552193 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.552136    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 19 10:35:15.552193 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.552157    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 19 10:35:15.556684 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.552950    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rcqd6\""
Sep 19 10:35:15.556684 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.554765    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-66fmf"]
Sep 19 10:35:15.579727 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.579700    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:15.606904 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.606885    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-jq5z7"]
Sep 19 10:35:15.612922 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.612906    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-jq5z7"
Sep 19 10:35:15.615244 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.615227    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-b64qz\""
Sep 19 10:35:15.615387 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.615370    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 19 10:35:15.615473 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.615461    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 19 10:35:15.622968 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.622949    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-jq5z7"]
Sep 19 10:35:15.669046 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.669020    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e8659512-e949-4ab4-a636-7d0e84f95048-crio-socket\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:15.669152 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.669062    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktx76\" (UniqueName: \"kubernetes.io/projected/e8659512-e949-4ab4-a636-7d0e84f95048-kube-api-access-ktx76\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:15.669212 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.669153    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8659512-e949-4ab4-a636-7d0e84f95048-data-volume\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:15.669268 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.669206    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:15.669268 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.669234    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:15.670394 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.670375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:15.673997 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.673978    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q7nq7"]
Sep 19 10:35:15.677344 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:15.677321    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b45f9d0_487d_41c2_97e0_98865a07f066.slice/crio-0cb9acd3f7134dd4fe827658fa87bcba2afcbecda48341882b432a21d78dcf46 WatchSource:0}: Error finding container 0cb9acd3f7134dd4fe827658fa87bcba2afcbecda48341882b432a21d78dcf46: Status 404 returned error can't find the container with id 0cb9acd3f7134dd4fe827658fa87bcba2afcbecda48341882b432a21d78dcf46
Sep 19 10:35:15.767546 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.767503    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:15.769492 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.769472    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:15.769562 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.769501    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:15.769562 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.769533    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e8659512-e949-4ab4-a636-7d0e84f95048-crio-socket\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:15.769562 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.769555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktx76\" (UniqueName: \"kubernetes.io/projected/e8659512-e949-4ab4-a636-7d0e84f95048-kube-api-access-ktx76\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:15.769681 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:15.769604    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 10:35:15.769681 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.769628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8659512-e949-4ab4-a636-7d0e84f95048-data-volume\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:15.769681 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:15.769634    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 10:35:15.769681 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.769649    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msnp4\" (UniqueName: \"kubernetes.io/projected/98e60d0a-95a5-4f4a-9b82-64148a71fe7a-kube-api-access-msnp4\") pod \"migrator-76794cbd6c-jq5z7\" (UID: \"98e60d0a-95a5-4f4a-9b82-64148a71fe7a\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-jq5z7"
Sep 19 10:35:15.769681 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:15.769672    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm podName:e8659512-e949-4ab4-a636-7d0e84f95048 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:16.269654532 +0000 UTC m=+37.181348042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm") pod "insights-runtime-extractor-66fmf" (UID: "e8659512-e949-4ab4-a636-7d0e84f95048") : configmap "kube-rbac-proxy" not found
Sep 19 10:35:15.769959 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:15.769703    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls podName:e8659512-e949-4ab4-a636-7d0e84f95048 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:16.269691507 +0000 UTC m=+37.181385005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls") pod "insights-runtime-extractor-66fmf" (UID: "e8659512-e949-4ab4-a636-7d0e84f95048") : secret "insights-runtime-extractor-tls" not found
Sep 19 10:35:15.769959 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.769817    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e8659512-e949-4ab4-a636-7d0e84f95048-crio-socket\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:15.770064 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.769987    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8659512-e949-4ab4-a636-7d0e84f95048-data-volume\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:15.781462 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.781440    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktx76\" (UniqueName: \"kubernetes.io/projected/e8659512-e949-4ab4-a636-7d0e84f95048-kube-api-access-ktx76\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:15.870447 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.870390    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msnp4\" (UniqueName: \"kubernetes.io/projected/98e60d0a-95a5-4f4a-9b82-64148a71fe7a-kube-api-access-msnp4\") pod \"migrator-76794cbd6c-jq5z7\" (UID: \"98e60d0a-95a5-4f4a-9b82-64148a71fe7a\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-jq5z7"
Sep 19 10:35:15.878040 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.878012    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msnp4\" (UniqueName: \"kubernetes.io/projected/98e60d0a-95a5-4f4a-9b82-64148a71fe7a-kube-api-access-msnp4\") pod \"migrator-76794cbd6c-jq5z7\" (UID: \"98e60d0a-95a5-4f4a-9b82-64148a71fe7a\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-jq5z7"
Sep 19 10:35:15.921081 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:15.921061    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-jq5z7"
Sep 19 10:35:16.109266 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.109032    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:16.109489 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.109050    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:35:16.109489 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.109054    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:35:16.112190 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.112168    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 10:35:16.112283 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.112168    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8jxmp\""
Sep 19 10:35:16.112283 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.112166    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 10:35:16.112398 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.112178    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 10:35:16.113358 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.113342    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d9kv4\""
Sep 19 10:35:16.113455 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.113381    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 10:35:16.204007 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.203980    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-jq5z7"]
Sep 19 10:35:16.207093 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:16.207070    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98e60d0a_95a5_4f4a_9b82_64148a71fe7a.slice/crio-9514bd25476327a46100207eade4c6d81776a79456d95a9840fdb1cf277f6cf4 WatchSource:0}: Error finding container 9514bd25476327a46100207eade4c6d81776a79456d95a9840fdb1cf277f6cf4: Status 404 returned error can't find the container with id 9514bd25476327a46100207eade4c6d81776a79456d95a9840fdb1cf277f6cf4
Sep 19 10:35:16.271514 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.271480    2579 generic.go:356] "Generic (PLEG): container finished" podID="fd50bc98-a748-4587-aacf-56daf296fe88" containerID="7a73f1ea6c5c4fd90e6449cc0f135f260559544a13295573b437d3f31a539506" exitCode=0
Sep 19 10:35:16.271693 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.271537    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p4shs" event={"ID":"fd50bc98-a748-4587-aacf-56daf296fe88","Type":"ContainerDied","Data":"7a73f1ea6c5c4fd90e6449cc0f135f260559544a13295573b437d3f31a539506"}
Sep 19 10:35:16.272770 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.272704    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:16.272770 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.272739    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:16.272971 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.272870    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-jq5z7" event={"ID":"98e60d0a-95a5-4f4a-9b82-64148a71fe7a","Type":"ContainerStarted","Data":"9514bd25476327a46100207eade4c6d81776a79456d95a9840fdb1cf277f6cf4"}
Sep 19 10:35:16.273181 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:16.272983    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 10:35:16.273181 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:16.273053    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm podName:e8659512-e949-4ab4-a636-7d0e84f95048 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:17.273033986 +0000 UTC m=+38.184727497 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm") pod "insights-runtime-extractor-66fmf" (UID: "e8659512-e949-4ab4-a636-7d0e84f95048") : configmap "kube-rbac-proxy" not found
Sep 19 10:35:16.273357 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:16.273237    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 10:35:16.273357 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:16.273292    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls podName:e8659512-e949-4ab4-a636-7d0e84f95048 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:17.2732724 +0000 UTC m=+38.184965898 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls") pod "insights-runtime-extractor-66fmf" (UID: "e8659512-e949-4ab4-a636-7d0e84f95048") : secret "insights-runtime-extractor-tls" not found
Sep 19 10:35:16.274410 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.274378    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q7nq7" event={"ID":"0b45f9d0-487d-41c2-97e0-98865a07f066","Type":"ContainerStarted","Data":"0cb9acd3f7134dd4fe827658fa87bcba2afcbecda48341882b432a21d78dcf46"}
Sep 19 10:35:16.374207 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.374177    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:16.374311 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:16.374283    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 10:35:16.374366 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:16.374346    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls podName:d6349d68-d16e-4a28-b8c1-40630155ed16 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:18.374326528 +0000 UTC m=+39.286020044 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls") pod "dns-default-jqhh6" (UID: "d6349d68-d16e-4a28-b8c1-40630155ed16") : secret "dns-default-metrics-tls" not found
Sep 19 10:35:16.541316 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.541284    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:16.605567 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:16.605536    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:17.232246 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:17.232211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:17.280559 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:17.280511    2579 generic.go:356] "Generic (PLEG): container finished" podID="fd50bc98-a748-4587-aacf-56daf296fe88" containerID="994ca3c7b0a229272ef35bcd9cee01fe36b6aea442dbcaedde7139b409de3c1b" exitCode=0
Sep 19 10:35:17.280752 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:17.280566    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p4shs" event={"ID":"fd50bc98-a748-4587-aacf-56daf296fe88","Type":"ContainerDied","Data":"994ca3c7b0a229272ef35bcd9cee01fe36b6aea442dbcaedde7139b409de3c1b"}
Sep 19 10:35:17.283664 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:17.283641    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:17.283786 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:17.283677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:17.283786 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:17.283769    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 10:35:17.283890 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:17.283838    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 10:35:17.283890 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:17.283844    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm podName:e8659512-e949-4ab4-a636-7d0e84f95048 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:19.28382448 +0000 UTC m=+40.195517990 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm") pod "insights-runtime-extractor-66fmf" (UID: "e8659512-e949-4ab4-a636-7d0e84f95048") : configmap "kube-rbac-proxy" not found
Sep 19 10:35:17.283990 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:17.283901    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls podName:e8659512-e949-4ab4-a636-7d0e84f95048 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:19.283884753 +0000 UTC m=+40.195578250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls") pod "insights-runtime-extractor-66fmf" (UID: "e8659512-e949-4ab4-a636-7d0e84f95048") : secret "insights-runtime-extractor-tls" not found
Sep 19 10:35:17.306275 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:17.306250    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:17.693220 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:17.693187    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wr84k_5492514c-c9bf-4516-9314-d1adda92b9ff/dns-node-resolver/0.log"
Sep 19 10:35:18.260130 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:18.259915    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:18.286052 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:18.286023    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p4shs" event={"ID":"fd50bc98-a748-4587-aacf-56daf296fe88","Type":"ContainerStarted","Data":"55f111647b9dbf6e036e55b0d47e63f30800c7ca9c82521ad2562df6082458c2"}
Sep 19 10:35:18.287799 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:18.287774    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-jq5z7" event={"ID":"98e60d0a-95a5-4f4a-9b82-64148a71fe7a","Type":"ContainerStarted","Data":"77973894e42ee7f161b4595c6330994bbc9d0444162764ab7cd2a5a64e96cbdb"}
Sep 19 10:35:18.287930 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:18.287826    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-jq5z7" event={"ID":"98e60d0a-95a5-4f4a-9b82-64148a71fe7a","Type":"ContainerStarted","Data":"267a48ad15fb26ef56c67d37838775c9fd9e542eee15fd9d3924fe2775beed67"}
Sep 19 10:35:18.289192 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:18.289167    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q7nq7" event={"ID":"0b45f9d0-487d-41c2-97e0-98865a07f066","Type":"ContainerStarted","Data":"8767f2387b7fa77a56610355c860561e56c1a1ef4a422747d13b7f918d859fc9"}
Sep 19 10:35:18.320966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:18.320921    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p4shs" podStartSLOduration=9.289221994 podStartE2EDuration="37.320909442s" totalImagesPullingTime="28.031687448s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:34:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:18.319254392 +0000 UTC m=+39.230947912" watchObservedRunningTime="2026-09-19 10:35:18.320909442 +0000 UTC m=+39.232602955"
Sep 19 10:35:18.341036 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:18.340995    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q7nq7" podStartSLOduration=2.066247058 podStartE2EDuration="4.340983458s" totalImagesPullingTime="2.2747364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:35:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:18.339826674 +0000 UTC m=+39.251520193" watchObservedRunningTime="2026-09-19 10:35:18.340983458 +0000 UTC m=+39.252676976"
Sep 19 10:35:18.361742 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:18.361705    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-jq5z7" podStartSLOduration=1.621979625 podStartE2EDuration="3.361693715s" totalImagesPullingTime="1.73971409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:18.360512362 +0000 UTC m=+39.272205880" watchObservedRunningTime="2026-09-19 10:35:18.361693715 +0000 UTC m=+39.273387266"
Sep 19 10:35:18.393352 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:18.393332    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:18.393624 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:18.393601    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 10:35:18.393723 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:18.393666    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls podName:d6349d68-d16e-4a28-b8c1-40630155ed16 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:22.393650281 +0000 UTC m=+43.305343795 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls") pod "dns-default-jqhh6" (UID: "d6349d68-d16e-4a28-b8c1-40630155ed16") : secret "dns-default-metrics-tls" not found
Sep 19 10:35:18.494013 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:18.493988    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qrxfh_2c21f7fa-a5d2-4c47-9e12-7ee4a905e870/node-ca/0.log"
Sep 19 10:35:18.606671 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:18.606598    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:18.682945 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:18.682917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:19.300245 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:19.300218    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:19.300245 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:19.300249    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:19.300847 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:19.300338    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 10:35:19.300847 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:19.300349    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 10:35:19.300847 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:19.300397    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls podName:e8659512-e949-4ab4-a636-7d0e84f95048 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:23.300384955 +0000 UTC m=+44.212078466 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls") pod "insights-runtime-extractor-66fmf" (UID: "e8659512-e949-4ab4-a636-7d0e84f95048") : secret "insights-runtime-extractor-tls" not found
Sep 19 10:35:19.300847 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:19.300411    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm podName:e8659512-e949-4ab4-a636-7d0e84f95048 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:23.300403345 +0000 UTC m=+44.212096842 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm") pod "insights-runtime-extractor-66fmf" (UID: "e8659512-e949-4ab4-a636-7d0e84f95048") : configmap "kube-rbac-proxy" not found
Sep 19 10:35:19.494395 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:19.494371    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q7nq7_0b45f9d0-487d-41c2-97e0-98865a07f066/serve-healthcheck-canary/0.log"
Sep 19 10:35:20.121233 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:20.121207    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:20.659126 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:20.659092    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:20.924137 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:20.924113    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:21.024975 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:21.024947    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:21.129709 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:21.129680    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:21.235451 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:21.235386    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:21.361426 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:21.361403    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:22.379910 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:22.379880    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:22.423054 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:22.423027    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:22.423195 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:22.423181    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 10:35:22.423259 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:22.423250    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls podName:d6349d68-d16e-4a28-b8c1-40630155ed16 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:30.423232492 +0000 UTC m=+51.334926001 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls") pod "dns-default-jqhh6" (UID: "d6349d68-d16e-4a28-b8c1-40630155ed16") : secret "dns-default-metrics-tls" not found
Sep 19 10:35:22.556753 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:22.556726    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:22.961938 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:22.961908    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:23.160241 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:23.160201    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:23.329874 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:23.329801    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:23.329874 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:23.329833    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:23.330044 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:23.329938    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 10:35:23.330044 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:23.330001    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls podName:e8659512-e949-4ab4-a636-7d0e84f95048 nodeName:}" failed. No retries permitted until 2026-09-19 10:35:31.329984538 +0000 UTC m=+52.241678037 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls") pod "insights-runtime-extractor-66fmf" (UID: "e8659512-e949-4ab4-a636-7d0e84f95048") : secret "insights-runtime-extractor-tls" not found
Sep 19 10:35:23.330337 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:23.330316    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8659512-e949-4ab4-a636-7d0e84f95048-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:23.756443 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:23.756413    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:24.159070 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:24.158990    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:24.362211 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:24.362180    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:24.477997 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:24.477968    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:24.548195 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:24.548171    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:24.606424 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:24.606402    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:24.756357 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:24.756297    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:25.381168 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:25.381105    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:25.443425 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:25.443399    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:25.562100 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:25.562071    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:25.960157 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:25.960127    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:26.558157 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:26.558129    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:26.954684 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:26.954656    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:27.162403 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:27.162374    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:27.361127 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:27.361057    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:27.754999 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:27.754970    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:28.027865 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:28.027797    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:28.560266 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:28.560239    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:28.955493 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:28.955462    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:29.368657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:29.368590    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:29.757283 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:29.757252    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:29.957582 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:29.957553    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:30.160180 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:30.160111    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:30.482803 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:30.482775    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:30.486539 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:30.486501    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6349d68-d16e-4a28-b8c1-40630155ed16-metrics-tls\") pod \"dns-default-jqhh6\" (UID: \"d6349d68-d16e-4a28-b8c1-40630155ed16\") " pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:30.521049 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:30.521026    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:30.828494 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:30.828425    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jqhh6"]
Sep 19 10:35:30.831802 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:30.831776    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6349d68_d16e_4a28_b8c1_40630155ed16.slice/crio-af883b4d13afe30d66110a515dfbd26b8693d37729e9a47772c0251050477ff8 WatchSource:0}: Error finding container af883b4d13afe30d66110a515dfbd26b8693d37729e9a47772c0251050477ff8: Status 404 returned error can't find the container with id af883b4d13afe30d66110a515dfbd26b8693d37729e9a47772c0251050477ff8
Sep 19 10:35:30.967919 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:30.967876    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:31.157048 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:31.156972    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:31.318260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:31.318225    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jqhh6" event={"ID":"d6349d68-d16e-4a28-b8c1-40630155ed16","Type":"ContainerStarted","Data":"af883b4d13afe30d66110a515dfbd26b8693d37729e9a47772c0251050477ff8"}
Sep 19 10:35:31.359494 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:31.359466    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:31.387510 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:31.387484    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:31.390111 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:31.390086    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8659512-e949-4ab4-a636-7d0e84f95048-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-66fmf\" (UID: \"e8659512-e949-4ab4-a636-7d0e84f95048\") " pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:31.466590 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:31.466563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-66fmf"
Sep 19 10:35:31.754045 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:31.753973    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-66fmf"]
Sep 19 10:35:31.757972 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:31.757944    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8659512_e949_4ab4_a636_7d0e84f95048.slice/crio-0869e6657ee00e9a64485c225724be4825cbff7870443730cbfa474d964a63fa WatchSource:0}: Error finding container 0869e6657ee00e9a64485c225724be4825cbff7870443730cbfa474d964a63fa: Status 404 returned error can't find the container with id 0869e6657ee00e9a64485c225724be4825cbff7870443730cbfa474d964a63fa
Sep 19 10:35:32.322300 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:32.322264    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-66fmf" event={"ID":"e8659512-e949-4ab4-a636-7d0e84f95048","Type":"ContainerStarted","Data":"4143f7ff368491d309c93f6a7f7bde4df85c0685f571a04f30ed84c3296ad9e1"}
Sep 19 10:35:32.322676 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:32.322330    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-66fmf" event={"ID":"e8659512-e949-4ab4-a636-7d0e84f95048","Type":"ContainerStarted","Data":"0869e6657ee00e9a64485c225724be4825cbff7870443730cbfa474d964a63fa"}
Sep 19 10:35:33.327007 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:33.326976    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jqhh6" event={"ID":"d6349d68-d16e-4a28-b8c1-40630155ed16","Type":"ContainerStarted","Data":"5ffb110207bfc1e88cc84cadf0ec231c3e64e90596876ed992e4ce92095aa43f"}
Sep 19 10:35:33.327375 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:33.327051    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jqhh6" event={"ID":"d6349d68-d16e-4a28-b8c1-40630155ed16","Type":"ContainerStarted","Data":"dff4cc37b9764f55f26ab3dded381c553870a8764dd42d1e40b738b43e987efe"}
Sep 19 10:35:33.327375 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:33.327105    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:33.346002 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:33.345963    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jqhh6" podStartSLOduration=17.337682929 podStartE2EDuration="19.345951142s" totalImagesPullingTime="2.008268213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:35:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:33.345000621 +0000 UTC m=+54.256694140" watchObservedRunningTime="2026-09-19 10:35:33.345951142 +0000 UTC m=+54.257644718"
Sep 19 10:35:34.332347 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:34.332313    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-66fmf" event={"ID":"e8659512-e949-4ab4-a636-7d0e84f95048","Type":"ContainerStarted","Data":"1a0142ef4e72f185aea1ad50f188c53864dab60de06ac5dd23ea22395a66d127"}
Sep 19 10:35:35.084596 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:35.084566    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:35.183816 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:35.183780    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:35.336965 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:35.336889    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-66fmf" event={"ID":"e8659512-e949-4ab4-a636-7d0e84f95048","Type":"ContainerStarted","Data":"51768b3b8ca250ecc08110d34187d23cb7842ffe921d64cfec9bf702c3b6da09"}
Sep 19 10:35:35.359286 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:35.359246    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-66fmf" podStartSLOduration=17.272091743 podStartE2EDuration="20.359228746s" totalImagesPullingTime="3.087137003s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:35:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:35.358746375 +0000 UTC m=+56.270439894" watchObservedRunningTime="2026-09-19 10:35:35.359228746 +0000 UTC m=+56.270922268"
Sep 19 10:35:37.346926 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.346895    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66d5748456-tv79x"]
Sep 19 10:35:37.371411 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.371383    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"]
Sep 19 10:35:37.371577 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.371504    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66d5748456-tv79x"
Sep 19 10:35:37.374609 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.374583    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 19 10:35:37.374747 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.374639    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 19 10:35:37.376124 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.376102    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-vjznz\""
Sep 19 10:35:37.376230 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.376162    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 19 10:35:37.376230 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.376168    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 19 10:35:37.386732 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.386714    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"
Sep 19 10:35:37.386732 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.386727    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66d5748456-tv79x"]
Sep 19 10:35:37.386854 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.386753    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"]
Sep 19 10:35:37.386854 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.386771    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56b557bc79-sqmkg"]
Sep 19 10:35:37.389601 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.389587    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 19 10:35:37.406684 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.406668    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56b557bc79-sqmkg"]
Sep 19 10:35:37.406761 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.406743    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.409611 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.409596    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-59nw8\""
Sep 19 10:35:37.409908 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.409891    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 19 10:35:37.409908 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.409892    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 19 10:35:37.410025 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.409892    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 19 10:35:37.415168 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.415152    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 19 10:35:37.431932 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.431906    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5d5w\" (UniqueName: \"kubernetes.io/projected/671da0f4-06ca-4e0d-878b-8be9a9e5cfda-kube-api-access-b5d5w\") pod \"managed-serviceaccount-addon-agent-66d5748456-tv79x\" (UID: \"671da0f4-06ca-4e0d-878b-8be9a9e5cfda\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66d5748456-tv79x"
Sep 19 10:35:37.432025 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.431931    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d54dedf1-5f91-45c6-b218-5d2d342a962e-bound-sa-token\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.432025 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.431959    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d54dedf1-5f91-45c6-b218-5d2d342a962e-ca-trust-extracted\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.432025 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.431992    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m62x\" (UniqueName: \"kubernetes.io/projected/d54dedf1-5f91-45c6-b218-5d2d342a962e-kube-api-access-5m62x\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.432167 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.432035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1b553b2f-2397-4c5b-8faa-38a9e7f82fe3-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cd88cc8bd-vl8vz\" (UID: \"1b553b2f-2397-4c5b-8faa-38a9e7f82fe3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"
Sep 19 10:35:37.432167 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.432079    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d54dedf1-5f91-45c6-b218-5d2d342a962e-image-registry-private-configuration\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.432167 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.432116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d54dedf1-5f91-45c6-b218-5d2d342a962e-registry-certificates\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.432167 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.432132    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d54dedf1-5f91-45c6-b218-5d2d342a962e-installation-pull-secrets\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.432167 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.432146    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b553b2f-2397-4c5b-8faa-38a9e7f82fe3-tmp\") pod \"klusterlet-addon-workmgr-5cd88cc8bd-vl8vz\" (UID: \"1b553b2f-2397-4c5b-8faa-38a9e7f82fe3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"
Sep 19 10:35:37.432327 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.432168    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/671da0f4-06ca-4e0d-878b-8be9a9e5cfda-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66d5748456-tv79x\" (UID: \"671da0f4-06ca-4e0d-878b-8be9a9e5cfda\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66d5748456-tv79x"
Sep 19 10:35:37.432327 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.432195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d54dedf1-5f91-45c6-b218-5d2d342a962e-trusted-ca\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.432327 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.432211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9692\" (UniqueName: \"kubernetes.io/projected/1b553b2f-2397-4c5b-8faa-38a9e7f82fe3-kube-api-access-w9692\") pod \"klusterlet-addon-workmgr-5cd88cc8bd-vl8vz\" (UID: \"1b553b2f-2397-4c5b-8faa-38a9e7f82fe3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"
Sep 19 10:35:37.432327 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.432233    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d54dedf1-5f91-45c6-b218-5d2d342a962e-registry-tls\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.497622 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.497595    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:37.532654 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.532633    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/671da0f4-06ca-4e0d-878b-8be9a9e5cfda-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66d5748456-tv79x\" (UID: \"671da0f4-06ca-4e0d-878b-8be9a9e5cfda\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66d5748456-tv79x"
Sep 19 10:35:37.532742 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.532658    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d54dedf1-5f91-45c6-b218-5d2d342a962e-trusted-ca\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.532742 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.532678    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9692\" (UniqueName: \"kubernetes.io/projected/1b553b2f-2397-4c5b-8faa-38a9e7f82fe3-kube-api-access-w9692\") pod \"klusterlet-addon-workmgr-5cd88cc8bd-vl8vz\" (UID: \"1b553b2f-2397-4c5b-8faa-38a9e7f82fe3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"
Sep 19 10:35:37.532742 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.532696    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d54dedf1-5f91-45c6-b218-5d2d342a962e-registry-tls\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.532900 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.532757    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5d5w\" (UniqueName: \"kubernetes.io/projected/671da0f4-06ca-4e0d-878b-8be9a9e5cfda-kube-api-access-b5d5w\") pod \"managed-serviceaccount-addon-agent-66d5748456-tv79x\" (UID: \"671da0f4-06ca-4e0d-878b-8be9a9e5cfda\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66d5748456-tv79x"
Sep 19 10:35:37.532900 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.532781    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d54dedf1-5f91-45c6-b218-5d2d342a962e-bound-sa-token\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.532900 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.532814    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d54dedf1-5f91-45c6-b218-5d2d342a962e-ca-trust-extracted\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.532900 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.532841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m62x\" (UniqueName: \"kubernetes.io/projected/d54dedf1-5f91-45c6-b218-5d2d342a962e-kube-api-access-5m62x\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.532900 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.532862    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1b553b2f-2397-4c5b-8faa-38a9e7f82fe3-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cd88cc8bd-vl8vz\" (UID: \"1b553b2f-2397-4c5b-8faa-38a9e7f82fe3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"
Sep 19 10:35:37.532900 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.532889    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d54dedf1-5f91-45c6-b218-5d2d342a962e-image-registry-private-configuration\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.533256 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.532964    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d54dedf1-5f91-45c6-b218-5d2d342a962e-registry-certificates\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.533256 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.532989    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d54dedf1-5f91-45c6-b218-5d2d342a962e-installation-pull-secrets\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.533256 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.533011    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b553b2f-2397-4c5b-8faa-38a9e7f82fe3-tmp\") pod \"klusterlet-addon-workmgr-5cd88cc8bd-vl8vz\" (UID: \"1b553b2f-2397-4c5b-8faa-38a9e7f82fe3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"
Sep 19 10:35:37.533402 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.533280    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d54dedf1-5f91-45c6-b218-5d2d342a962e-ca-trust-extracted\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.533402 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.533389    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b553b2f-2397-4c5b-8faa-38a9e7f82fe3-tmp\") pod \"klusterlet-addon-workmgr-5cd88cc8bd-vl8vz\" (UID: \"1b553b2f-2397-4c5b-8faa-38a9e7f82fe3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"
Sep 19 10:35:37.533837 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.533815    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d54dedf1-5f91-45c6-b218-5d2d342a962e-trusted-ca\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.533966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.533837    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d54dedf1-5f91-45c6-b218-5d2d342a962e-registry-certificates\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.535549 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.535505    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d54dedf1-5f91-45c6-b218-5d2d342a962e-image-registry-private-configuration\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.535685 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.535662    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d54dedf1-5f91-45c6-b218-5d2d342a962e-registry-tls\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.535774 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.535738    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d54dedf1-5f91-45c6-b218-5d2d342a962e-installation-pull-secrets\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.535858 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.535842    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/671da0f4-06ca-4e0d-878b-8be9a9e5cfda-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-66d5748456-tv79x\" (UID: \"671da0f4-06ca-4e0d-878b-8be9a9e5cfda\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66d5748456-tv79x"
Sep 19 10:35:37.536204 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.536185    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1b553b2f-2397-4c5b-8faa-38a9e7f82fe3-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cd88cc8bd-vl8vz\" (UID: \"1b553b2f-2397-4c5b-8faa-38a9e7f82fe3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"
Sep 19 10:35:37.545108 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.545073    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5d5w\" (UniqueName: \"kubernetes.io/projected/671da0f4-06ca-4e0d-878b-8be9a9e5cfda-kube-api-access-b5d5w\") pod \"managed-serviceaccount-addon-agent-66d5748456-tv79x\" (UID: \"671da0f4-06ca-4e0d-878b-8be9a9e5cfda\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66d5748456-tv79x"
Sep 19 10:35:37.545191 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.545147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m62x\" (UniqueName: \"kubernetes.io/projected/d54dedf1-5f91-45c6-b218-5d2d342a962e-kube-api-access-5m62x\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.545393 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.545376    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d54dedf1-5f91-45c6-b218-5d2d342a962e-bound-sa-token\") pod \"image-registry-56b557bc79-sqmkg\" (UID: \"d54dedf1-5f91-45c6-b218-5d2d342a962e\") " pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.545468 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.545451    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9692\" (UniqueName: \"kubernetes.io/projected/1b553b2f-2397-4c5b-8faa-38a9e7f82fe3-kube-api-access-w9692\") pod \"klusterlet-addon-workmgr-5cd88cc8bd-vl8vz\" (UID: \"1b553b2f-2397-4c5b-8faa-38a9e7f82fe3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"
Sep 19 10:35:37.598101 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.598038    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:37.693458 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.693431    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66d5748456-tv79x"
Sep 19 10:35:37.701596 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.701577    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"
Sep 19 10:35:37.708625 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.708601    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:37.714581 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.714558    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:37.797731 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.797658    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:37.991760 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:37.991678    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66d5748456-tv79x"]
Sep 19 10:35:37.994226 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:37.994200    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod671da0f4_06ca_4e0d_878b_8be9a9e5cfda.slice/crio-68c075dc8bd56c9735442832269678be876c49588fa3c7a314c0338dd2fac3ba WatchSource:0}: Error finding container 68c075dc8bd56c9735442832269678be876c49588fa3c7a314c0338dd2fac3ba: Status 404 returned error can't find the container with id 68c075dc8bd56c9735442832269678be876c49588fa3c7a314c0338dd2fac3ba
Sep 19 10:35:38.018180 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:38.018157    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"]
Sep 19 10:35:38.020804 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:38.020782    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b553b2f_2397_4c5b_8faa_38a9e7f82fe3.slice/crio-484a689129bfaaa81f75575109eb265ee01cc330a0e03827432a3e45f4782713 WatchSource:0}: Error finding container 484a689129bfaaa81f75575109eb265ee01cc330a0e03827432a3e45f4782713: Status 404 returned error can't find the container with id 484a689129bfaaa81f75575109eb265ee01cc330a0e03827432a3e45f4782713
Sep 19 10:35:38.040102 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:38.040081    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56b557bc79-sqmkg"]
Sep 19 10:35:38.041506 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:38.041477    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd54dedf1_5f91_45c6_b218_5d2d342a962e.slice/crio-3750e6a9ac323146fd2900fd11805b5087755e2ccfed9d559bc76452b3b5758c WatchSource:0}: Error finding container 3750e6a9ac323146fd2900fd11805b5087755e2ccfed9d559bc76452b3b5758c: Status 404 returned error can't find the container with id 3750e6a9ac323146fd2900fd11805b5087755e2ccfed9d559bc76452b3b5758c
Sep 19 10:35:38.349445 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:38.349372    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66d5748456-tv79x" event={"ID":"671da0f4-06ca-4e0d-878b-8be9a9e5cfda","Type":"ContainerStarted","Data":"68c075dc8bd56c9735442832269678be876c49588fa3c7a314c0338dd2fac3ba"}
Sep 19 10:35:38.350574 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:38.350551    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz" event={"ID":"1b553b2f-2397-4c5b-8faa-38a9e7f82fe3","Type":"ContainerStarted","Data":"484a689129bfaaa81f75575109eb265ee01cc330a0e03827432a3e45f4782713"}
Sep 19 10:35:38.351960 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:38.351935    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56b557bc79-sqmkg" event={"ID":"d54dedf1-5f91-45c6-b218-5d2d342a962e","Type":"ContainerStarted","Data":"0e9a6b9bdfcb037e3518fc68e971d902deef39aee5f3b6e3752d648788b60127"}
Sep 19 10:35:38.352063 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:38.351981    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56b557bc79-sqmkg" event={"ID":"d54dedf1-5f91-45c6-b218-5d2d342a962e","Type":"ContainerStarted","Data":"3750e6a9ac323146fd2900fd11805b5087755e2ccfed9d559bc76452b3b5758c"}
Sep 19 10:35:38.352063 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:38.352019    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:38.371856 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:38.371788    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56b557bc79-sqmkg" podStartSLOduration=1.371769772 podStartE2EDuration="1.371769772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:35:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:38.371381521 +0000 UTC m=+59.283075039" watchObservedRunningTime="2026-09-19 10:35:38.371769772 +0000 UTC m=+59.283463293"
Sep 19 10:35:39.267282 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:39.267249    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wzspq"
Sep 19 10:35:39.405626 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:39.405596    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:39.476765 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:39.476733    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:39.637733 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:39.637620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:39.738552 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:39.738487    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:40.038905 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:40.038871    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:40.137593 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:40.137558    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:41.135942 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:41.135911    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:41.436123 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:41.436084    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:41.536666 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:41.536639    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:42.871600 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:42.871566    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:42.951677 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:42.951642    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:43.076520 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:43.076491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:43.334485 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:43.334418    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jqhh6"
Sep 19 10:35:43.371978 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:43.371944    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66d5748456-tv79x" event={"ID":"671da0f4-06ca-4e0d-878b-8be9a9e5cfda","Type":"ContainerStarted","Data":"f13fc13774c8341395a39ece2b7e9f8dfb82d6cd89f4b853bf0828fa296d02e2"}
Sep 19 10:35:43.373558 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:43.373533    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz" event={"ID":"1b553b2f-2397-4c5b-8faa-38a9e7f82fe3","Type":"ContainerStarted","Data":"e3425f7c9363459f6f9507425dcf960414743340bcdbd9a96ad0d2dd00d955e9"}
Sep 19 10:35:43.373723 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:43.373710    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"
Sep 19 10:35:43.375020 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:43.375001    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz"
Sep 19 10:35:43.388411 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:43.388370    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-66d5748456-tv79x" podStartSLOduration=1.709339046 podStartE2EDuration="6.38835766s" totalImagesPullingTime="4.679018614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:43.388191706 +0000 UTC m=+64.299885227" watchObservedRunningTime="2026-09-19 10:35:43.38835766 +0000 UTC m=+64.300051176"
Sep 19 10:35:43.411844 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:43.411799    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cd88cc8bd-vl8vz" podStartSLOduration=1.7400761980000001 podStartE2EDuration="6.411781667s" totalImagesPullingTime="4.671705469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:43.410727927 +0000 UTC m=+64.322421446" watchObservedRunningTime="2026-09-19 10:35:43.411781667 +0000 UTC m=+64.323475189"
Sep 19 10:35:43.464017 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:43.463989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:43.528029 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:43.528002    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:46.100542 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:46.100493    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:46.176311 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:46.176274    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:46.256890 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:46.256861    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:46.808904 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:46.808870    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs\") pod \"network-metrics-daemon-lnqqz\" (UID: \"f85be02b-a097-4f90-844c-21d54e7dc53d\") " pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:35:46.811731 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:46.811710    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 10:35:46.822113 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:46.822084    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f85be02b-a097-4f90-844c-21d54e7dc53d-metrics-certs\") pod \"network-metrics-daemon-lnqqz\" (UID: \"f85be02b-a097-4f90-844c-21d54e7dc53d\") " pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:35:46.909445 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:46.909403    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:35:46.909594 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:46.909476    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878zs\" (UniqueName: \"kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs\") pod \"network-check-target-9f2kq\" (UID: \"4c9f2fc8-10e3-4bda-8b12-8f05918712f2\") " pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:46.912483 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:46.912466    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 10:35:46.912545 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:46.912495    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 10:35:46.921865 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:46.921845    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/630f71c6-b3e9-4935-8c99-921f92d7494c-original-pull-secret\") pod \"global-pull-secret-syncer-vr6wb\" (UID: \"630f71c6-b3e9-4935-8c99-921f92d7494c\") " pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:35:46.925419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:46.925398    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 10:35:46.932895 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:46.932877    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-878zs\" (UniqueName: \"kubernetes.io/projected/4c9f2fc8-10e3-4bda-8b12-8f05918712f2-kube-api-access-878zs\") pod \"network-check-target-9f2kq\" (UID: \"4c9f2fc8-10e3-4bda-8b12-8f05918712f2\") " pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:47.024976 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:47.024951    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-d9kv4\""
Sep 19 10:35:47.030509 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:47.030494    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8jxmp\""
Sep 19 10:35:47.032485 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:47.032470    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:47.034690 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:47.034671    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vr6wb"
Sep 19 10:35:47.039106 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:47.039090    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lnqqz"
Sep 19 10:35:47.314245 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:47.314211    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9f2kq"]
Sep 19 10:35:47.315311 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:47.315277    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lnqqz"]
Sep 19 10:35:47.316211 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:47.316126    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vr6wb"]
Sep 19 10:35:47.318278 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:47.318255    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf85be02b_a097_4f90_844c_21d54e7dc53d.slice/crio-2ed61ec32e41bb7005f2d36331fb6149a8b7ca837e8c944c7d53205d9fe1af7a WatchSource:0}: Error finding container 2ed61ec32e41bb7005f2d36331fb6149a8b7ca837e8c944c7d53205d9fe1af7a: Status 404 returned error can't find the container with id 2ed61ec32e41bb7005f2d36331fb6149a8b7ca837e8c944c7d53205d9fe1af7a
Sep 19 10:35:47.319014 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:47.318991    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c9f2fc8_10e3_4bda_8b12_8f05918712f2.slice/crio-1fbef73427a5320e52901d40004019eff5abc10d642e4bc8d70a1f3c62ba99aa WatchSource:0}: Error finding container 1fbef73427a5320e52901d40004019eff5abc10d642e4bc8d70a1f3c62ba99aa: Status 404 returned error can't find the container with id 1fbef73427a5320e52901d40004019eff5abc10d642e4bc8d70a1f3c62ba99aa
Sep 19 10:35:47.319650 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:47.319623    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod630f71c6_b3e9_4935_8c99_921f92d7494c.slice/crio-fac60287d5fd4e21ce08a801e829a12670418478e27c85e356b6aa294dd7eedc WatchSource:0}: Error finding container fac60287d5fd4e21ce08a801e829a12670418478e27c85e356b6aa294dd7eedc: Status 404 returned error can't find the container with id fac60287d5fd4e21ce08a801e829a12670418478e27c85e356b6aa294dd7eedc
Sep 19 10:35:47.388365 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:47.388295    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lnqqz" event={"ID":"f85be02b-a097-4f90-844c-21d54e7dc53d","Type":"ContainerStarted","Data":"2ed61ec32e41bb7005f2d36331fb6149a8b7ca837e8c944c7d53205d9fe1af7a"}
Sep 19 10:35:47.389413 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:47.389391    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9f2kq" event={"ID":"4c9f2fc8-10e3-4bda-8b12-8f05918712f2","Type":"ContainerStarted","Data":"1fbef73427a5320e52901d40004019eff5abc10d642e4bc8d70a1f3c62ba99aa"}
Sep 19 10:35:47.390460 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:47.390436    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vr6wb" event={"ID":"630f71c6-b3e9-4935-8c99-921f92d7494c","Type":"ContainerStarted","Data":"fac60287d5fd4e21ce08a801e829a12670418478e27c85e356b6aa294dd7eedc"}
Sep 19 10:35:47.610704 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:47.610674    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:47.712426 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:47.712378    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:47.766574 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:47.766538    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:48.240295 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:48.240264    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:49.400511 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:49.400472    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lnqqz" event={"ID":"f85be02b-a097-4f90-844c-21d54e7dc53d","Type":"ContainerStarted","Data":"a75d5b116ba493b3e20c3312ad21aee7435c92d88f424ef471854dd1d31191f0"}
Sep 19 10:35:49.400973 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:49.400558    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lnqqz" event={"ID":"f85be02b-a097-4f90-844c-21d54e7dc53d","Type":"ContainerStarted","Data":"2f3ee12bfd04c3c91d3d110ee32c7fd9afcc84750f8740e6063613e9e96cef86"}
Sep 19 10:35:49.418929 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:49.418886    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lnqqz" podStartSLOduration=67.390247547 podStartE2EDuration="1m8.418868465s" totalImagesPullingTime="1.028620918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:49.418559365 +0000 UTC m=+70.330252926" watchObservedRunningTime="2026-09-19 10:35:49.418868465 +0000 UTC m=+70.330561984"
Sep 19 10:35:49.522793 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:49.522758    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:49.623298 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:49.623271    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:50.099206 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.099172    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:50.239212 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.239179    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:50.344144 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.344112    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:50.445277 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.445246    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:50.586136 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.586101    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:50.624188 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.624159    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"]
Sep 19 10:35:50.629463 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.629442    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"
Sep 19 10:35:50.632299 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.632275    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 19 10:35:50.632445 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.632373    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 19 10:35:50.633936 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.633679    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 19 10:35:50.633936 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.633880    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zskmh\""
Sep 19 10:35:50.633936 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.633896    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 19 10:35:50.633936 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.633922    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 19 10:35:50.635155 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.635132    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gzxgf"]
Sep 19 10:35:50.637226 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.637210    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.639798 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.639780    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 19 10:35:50.639898 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.639844    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 19 10:35:50.640075 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.640009    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 19 10:35:50.640500 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.640484    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zsbvs\""
Sep 19 10:35:50.646636 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.646610    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"]
Sep 19 10:35:50.702060 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.701978    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:50.739269 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.739239    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.739420 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.739279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qfk4\" (UniqueName: \"kubernetes.io/projected/a0f9227b-ae29-4410-968b-f7e2ce416ffa-kube-api-access-8qfk4\") pod \"openshift-state-metrics-86c5bc79d6-lb88l\" (UID: \"a0f9227b-ae29-4410-968b-f7e2ce416ffa\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"
Sep 19 10:35:50.739420 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.739366    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-tls\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.739420 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.739389    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0f9227b-ae29-4410-968b-f7e2ce416ffa-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-lb88l\" (UID: \"a0f9227b-ae29-4410-968b-f7e2ce416ffa\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"
Sep 19 10:35:50.739420 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.739409    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aac09470-a013-4090-9983-ad5fb9f90b2a-root\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.739668 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.739435    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-accelerators-collector-config\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.739668 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.739575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-textfile\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.739668 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.739615    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aac09470-a013-4090-9983-ad5fb9f90b2a-metrics-client-ca\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.739823 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.739675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-wtmp\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.739823 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.739722    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0f9227b-ae29-4410-968b-f7e2ce416ffa-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-lb88l\" (UID: \"a0f9227b-ae29-4410-968b-f7e2ce416ffa\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"
Sep 19 10:35:50.739823 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.739755    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whtcl\" (UniqueName: \"kubernetes.io/projected/aac09470-a013-4090-9983-ad5fb9f90b2a-kube-api-access-whtcl\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.739823 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.739781    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aac09470-a013-4090-9983-ad5fb9f90b2a-sys\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.739823 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.739819    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0f9227b-ae29-4410-968b-f7e2ce416ffa-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-lb88l\" (UID: \"a0f9227b-ae29-4410-968b-f7e2ce416ffa\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"
Sep 19 10:35:50.779095 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.779060    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:50.840593 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.840558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-tls\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.840765 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.840603    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0f9227b-ae29-4410-968b-f7e2ce416ffa-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-lb88l\" (UID: \"a0f9227b-ae29-4410-968b-f7e2ce416ffa\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"
Sep 19 10:35:50.840765 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.840634    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aac09470-a013-4090-9983-ad5fb9f90b2a-root\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.840765 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.840659    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-accelerators-collector-config\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.840765 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.840689    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-textfile\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.840765 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.840715    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aac09470-a013-4090-9983-ad5fb9f90b2a-metrics-client-ca\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.840765 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:50.840718    2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 19 10:35:50.841067 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.840759    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aac09470-a013-4090-9983-ad5fb9f90b2a-root\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.841067 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:35:50.840801    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-tls podName:aac09470-a013-4090-9983-ad5fb9f90b2a nodeName:}" failed. No retries permitted until 2026-09-19 10:35:51.340780892 +0000 UTC m=+72.252474408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-tls") pod "node-exporter-gzxgf" (UID: "aac09470-a013-4090-9983-ad5fb9f90b2a") : secret "node-exporter-tls" not found
Sep 19 10:35:50.841067 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.841029    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-wtmp\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.841228 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.841069    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0f9227b-ae29-4410-968b-f7e2ce416ffa-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-lb88l\" (UID: \"a0f9227b-ae29-4410-968b-f7e2ce416ffa\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"
Sep 19 10:35:50.841228 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.841099    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whtcl\" (UniqueName: \"kubernetes.io/projected/aac09470-a013-4090-9983-ad5fb9f90b2a-kube-api-access-whtcl\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.841228 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.841129    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aac09470-a013-4090-9983-ad5fb9f90b2a-sys\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.841228 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.841151    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0f9227b-ae29-4410-968b-f7e2ce416ffa-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-lb88l\" (UID: \"a0f9227b-ae29-4410-968b-f7e2ce416ffa\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"
Sep 19 10:35:50.841228 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.841184    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.841228 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.841207    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qfk4\" (UniqueName: \"kubernetes.io/projected/a0f9227b-ae29-4410-968b-f7e2ce416ffa-kube-api-access-8qfk4\") pod \"openshift-state-metrics-86c5bc79d6-lb88l\" (UID: \"a0f9227b-ae29-4410-968b-f7e2ce416ffa\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"
Sep 19 10:35:50.841228 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.841218    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-wtmp\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.841432 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.841292    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aac09470-a013-4090-9983-ad5fb9f90b2a-metrics-client-ca\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.841432 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.841312    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aac09470-a013-4090-9983-ad5fb9f90b2a-sys\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.841432 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.841338    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-accelerators-collector-config\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.841856 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.841829    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-textfile\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.841983 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.841969    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0f9227b-ae29-4410-968b-f7e2ce416ffa-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-lb88l\" (UID: \"a0f9227b-ae29-4410-968b-f7e2ce416ffa\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"
Sep 19 10:35:50.843328 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.843302    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0f9227b-ae29-4410-968b-f7e2ce416ffa-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-lb88l\" (UID: \"a0f9227b-ae29-4410-968b-f7e2ce416ffa\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"
Sep 19 10:35:50.843956 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.843932    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.844080 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.844063    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0f9227b-ae29-4410-968b-f7e2ce416ffa-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-lb88l\" (UID: \"a0f9227b-ae29-4410-968b-f7e2ce416ffa\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"
Sep 19 10:35:50.850117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.850096    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whtcl\" (UniqueName: \"kubernetes.io/projected/aac09470-a013-4090-9983-ad5fb9f90b2a-kube-api-access-whtcl\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:50.850234 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.850216    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qfk4\" (UniqueName: \"kubernetes.io/projected/a0f9227b-ae29-4410-968b-f7e2ce416ffa-kube-api-access-8qfk4\") pod \"openshift-state-metrics-86c5bc79d6-lb88l\" (UID: \"a0f9227b-ae29-4410-968b-f7e2ce416ffa\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"
Sep 19 10:35:50.879516 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.879487    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:50.942416 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.942387    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"
Sep 19 10:35:50.980783 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:50.980713    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:51.084767 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:51.084727    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:51.179577 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:51.179542    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:51.345626 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:51.345508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-tls\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:51.347916 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:51.347886    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aac09470-a013-4090-9983-ad5fb9f90b2a-node-exporter-tls\") pod \"node-exporter-gzxgf\" (UID: \"aac09470-a013-4090-9983-ad5fb9f90b2a\") " pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:51.549708 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:51.549673    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gzxgf"
Sep 19 10:35:51.570223 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:51.570197    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:51.650248 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:51.650185    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:51.676105 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:51.676084    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaac09470_a013_4090_9983_ad5fb9f90b2a.slice/crio-e18acb2a2f821f004191970ab351faefba35ad309e1f3c095d29c6ec7435ce76 WatchSource:0}: Error finding container e18acb2a2f821f004191970ab351faefba35ad309e1f3c095d29c6ec7435ce76: Status 404 returned error can't find the container with id e18acb2a2f821f004191970ab351faefba35ad309e1f3c095d29c6ec7435ce76
Sep 19 10:35:51.943472 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:51.943440    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l"]
Sep 19 10:35:51.946353 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:51.946324    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0f9227b_ae29_4410_968b_f7e2ce416ffa.slice/crio-0b761782f08a675c559c53e8f68e6c2a7b926285eb7e2096128376875ccd0428 WatchSource:0}: Error finding container 0b761782f08a675c559c53e8f68e6c2a7b926285eb7e2096128376875ccd0428: Status 404 returned error can't find the container with id 0b761782f08a675c559c53e8f68e6c2a7b926285eb7e2096128376875ccd0428
Sep 19 10:35:52.108995 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:52.108937    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:52.412340 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:52.412264    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gzxgf" event={"ID":"aac09470-a013-4090-9983-ad5fb9f90b2a","Type":"ContainerStarted","Data":"e18acb2a2f821f004191970ab351faefba35ad309e1f3c095d29c6ec7435ce76"}
Sep 19 10:35:52.413761 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:52.413736    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9f2kq" event={"ID":"4c9f2fc8-10e3-4bda-8b12-8f05918712f2","Type":"ContainerStarted","Data":"448ca2fc010963865d733159a8a261f93801b41d002dd06f220e0c2b22fae6a5"}
Sep 19 10:35:52.413959 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:52.413785    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:35:52.415552 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:52.415515    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l" event={"ID":"a0f9227b-ae29-4410-968b-f7e2ce416ffa","Type":"ContainerStarted","Data":"1b5d95b15b3b971edd9126038eb9a78c325c7eb6dd0eb24fad2efe745c8dc3d7"}
Sep 19 10:35:52.415646 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:52.415574    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l" event={"ID":"a0f9227b-ae29-4410-968b-f7e2ce416ffa","Type":"ContainerStarted","Data":"1009ca2358911a61c153c40518f2571dc9ed21c18a55b44e93efa67094cc280a"}
Sep 19 10:35:52.415646 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:52.415594    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l" event={"ID":"a0f9227b-ae29-4410-968b-f7e2ce416ffa","Type":"ContainerStarted","Data":"0b761782f08a675c559c53e8f68e6c2a7b926285eb7e2096128376875ccd0428"}
Sep 19 10:35:52.416943 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:52.416922    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vr6wb" event={"ID":"630f71c6-b3e9-4935-8c99-921f92d7494c","Type":"ContainerStarted","Data":"160486c03a21b48e8d7a791440d8e1bf8755f6323afdf357ee596b715f276cb2"}
Sep 19 10:35:52.432204 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:52.432157    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9f2kq" podStartSLOduration=67.078035115 podStartE2EDuration="1m11.432140093s" totalImagesPullingTime="4.354104978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:52.429748246 +0000 UTC m=+73.341441786" watchObservedRunningTime="2026-09-19 10:35:52.432140093 +0000 UTC m=+73.343833613"
Sep 19 10:35:52.448858 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:52.448817    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vr6wb" podStartSLOduration=67.083417623 podStartE2EDuration="1m11.448805207s" totalImagesPullingTime="4.365387584s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:34:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:52.448488956 +0000 UTC m=+73.360182476" watchObservedRunningTime="2026-09-19 10:35:52.448805207 +0000 UTC m=+73.360498752"
Sep 19 10:35:53.422887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:53.422760    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l" event={"ID":"a0f9227b-ae29-4410-968b-f7e2ce416ffa","Type":"ContainerStarted","Data":"a02b0e74ce3c262d57a4a4924ce5885d44c07ab776a27fefb4ae51b7b161cb9e"}
Sep 19 10:35:53.424439 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:53.424415    2579 generic.go:356] "Generic (PLEG): container finished" podID="aac09470-a013-4090-9983-ad5fb9f90b2a" containerID="c84bcdf47b95da24d0a70ff50fc212edb6465ab41704e29ce2182d294fe660dc" exitCode=0
Sep 19 10:35:53.424559 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:53.424478    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gzxgf" event={"ID":"aac09470-a013-4090-9983-ad5fb9f90b2a","Type":"ContainerDied","Data":"c84bcdf47b95da24d0a70ff50fc212edb6465ab41704e29ce2182d294fe660dc"}
Sep 19 10:35:53.443335 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:53.443276    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-lb88l" podStartSLOduration=2.450278549 podStartE2EDuration="3.443258879s" totalImagesPullingTime="992.98033ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:35:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:53.442841463 +0000 UTC m=+74.354534983" watchObservedRunningTime="2026-09-19 10:35:53.443258879 +0000 UTC m=+74.354952399"
Sep 19 10:35:53.504342 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:53.504321    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:53.590056 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:53.590028    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:53.824365 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:53.824291    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:54.024882 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.024851    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:54.430998 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.430958    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gzxgf" event={"ID":"aac09470-a013-4090-9983-ad5fb9f90b2a","Type":"ContainerStarted","Data":"b8e1ea657ada24ad02c08a25860acabcf11bc84bf9d9096e267fc692bfb606ad"}
Sep 19 10:35:54.431431 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.431023    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gzxgf" event={"ID":"aac09470-a013-4090-9983-ad5fb9f90b2a","Type":"ContainerStarted","Data":"532f2027e7bf5148166b2f4f663cab3475692c4c9ab994c48c803d6768192b54"}
Sep 19 10:35:54.885025 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.884919    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gzxgf" podStartSLOduration=4.080360511 podStartE2EDuration="4.884896491s" totalImagesPullingTime="744.974886ms" totalInitContainerRuntime="59.561094ms" isStatefulPod=true podCreationTimestamp="2026-09-19 10:35:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:54.453123813 +0000 UTC m=+75.364817332" watchObservedRunningTime="2026-09-19 10:35:54.884896491 +0000 UTC m=+75.796590011"
Sep 19 10:35:54.885514 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.885499    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7674648dc8-9hb4n"]
Sep 19 10:35:54.887388 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.887374    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:54.890026 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.890003    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 19 10:35:54.890152 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.890068    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-zz9ll\""
Sep 19 10:35:54.890152 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.890105    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 19 10:35:54.891222 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.891207    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 19 10:35:54.891222 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.891213    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 19 10:35:54.891597 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.891582    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ft9a0fedgra1n\""
Sep 19 10:35:54.899946 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.899928    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7674648dc8-9hb4n"]
Sep 19 10:35:54.971482 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.971459    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpgnf\" (UniqueName: \"kubernetes.io/projected/47c4dd98-e96a-426c-a40b-821612719f6a-kube-api-access-dpgnf\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:54.971604 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.971490    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47c4dd98-e96a-426c-a40b-821612719f6a-client-ca-bundle\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:54.971604 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.971507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/47c4dd98-e96a-426c-a40b-821612719f6a-secret-metrics-server-tls\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:54.971604 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.971541    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/47c4dd98-e96a-426c-a40b-821612719f6a-secret-metrics-server-client-certs\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:54.971604 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.971586    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/47c4dd98-e96a-426c-a40b-821612719f6a-audit-log\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:54.971799 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.971613    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/47c4dd98-e96a-426c-a40b-821612719f6a-metrics-server-audit-profiles\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:54.971799 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:54.971706    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47c4dd98-e96a-426c-a40b-821612719f6a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.072611 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.072587    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47c4dd98-e96a-426c-a40b-821612719f6a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.072707 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.072622    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpgnf\" (UniqueName: \"kubernetes.io/projected/47c4dd98-e96a-426c-a40b-821612719f6a-kube-api-access-dpgnf\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.072707 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.072645    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47c4dd98-e96a-426c-a40b-821612719f6a-client-ca-bundle\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.072707 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.072664    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/47c4dd98-e96a-426c-a40b-821612719f6a-secret-metrics-server-tls\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.072707 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.072686    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/47c4dd98-e96a-426c-a40b-821612719f6a-secret-metrics-server-client-certs\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.072889 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.072708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/47c4dd98-e96a-426c-a40b-821612719f6a-audit-log\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.072889 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.072730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/47c4dd98-e96a-426c-a40b-821612719f6a-metrics-server-audit-profiles\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.074540 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.073417    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/47c4dd98-e96a-426c-a40b-821612719f6a-audit-log\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.074540 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.073949    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47c4dd98-e96a-426c-a40b-821612719f6a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.074540 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.074120    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/47c4dd98-e96a-426c-a40b-821612719f6a-metrics-server-audit-profiles\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.079186 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.079156    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/47c4dd98-e96a-426c-a40b-821612719f6a-secret-metrics-server-tls\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.079272 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.079161    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/47c4dd98-e96a-426c-a40b-821612719f6a-client-ca-bundle\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.080317 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.080298    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/47c4dd98-e96a-426c-a40b-821612719f6a-secret-metrics-server-client-certs\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.081011 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.080991    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpgnf\" (UniqueName: \"kubernetes.io/projected/47c4dd98-e96a-426c-a40b-821612719f6a-kube-api-access-dpgnf\") pod \"metrics-server-7674648dc8-9hb4n\" (UID: \"47c4dd98-e96a-426c-a40b-821612719f6a\") " pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.196115 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.196092    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:35:55.431514 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.431473    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:55.457163 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.457097    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr"]
Sep 19 10:35:55.459256 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.459237    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr"
Sep 19 10:35:55.461836 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.461814    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 19 10:35:55.461924 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.461873    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-cjvtr\""
Sep 19 10:35:55.469179 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.469160    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7674648dc8-9hb4n"]
Sep 19 10:35:55.471276 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:55.471251    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47c4dd98_e96a_426c_a40b_821612719f6a.slice/crio-d9a502a578c821536af0ad94d286969fa39f84c2699edf9d19863ad35053cf7f WatchSource:0}: Error finding container d9a502a578c821536af0ad94d286969fa39f84c2699edf9d19863ad35053cf7f: Status 404 returned error can't find the container with id d9a502a578c821536af0ad94d286969fa39f84c2699edf9d19863ad35053cf7f
Sep 19 10:35:55.472504 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.472487    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr"]
Sep 19 10:35:55.577665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.577639    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56fk9\" (UniqueName: \"kubernetes.io/projected/98fa1395-052d-4cb6-a5cc-e889c967b1a2-kube-api-access-56fk9\") pod \"monitoring-plugin-5984459ff8-sm5wr\" (UID: \"98fa1395-052d-4cb6-a5cc-e889c967b1a2\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr"
Sep 19 10:35:55.577784 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.577674    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/98fa1395-052d-4cb6-a5cc-e889c967b1a2-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-sm5wr\" (UID: \"98fa1395-052d-4cb6-a5cc-e889c967b1a2\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr"
Sep 19 10:35:55.645867 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.645842    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:55.678135 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.678110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56fk9\" (UniqueName: \"kubernetes.io/projected/98fa1395-052d-4cb6-a5cc-e889c967b1a2-kube-api-access-56fk9\") pod \"monitoring-plugin-5984459ff8-sm5wr\" (UID: \"98fa1395-052d-4cb6-a5cc-e889c967b1a2\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr"
Sep 19 10:35:55.678248 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.678144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/98fa1395-052d-4cb6-a5cc-e889c967b1a2-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-sm5wr\" (UID: \"98fa1395-052d-4cb6-a5cc-e889c967b1a2\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr"
Sep 19 10:35:55.680405 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.680388    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/98fa1395-052d-4cb6-a5cc-e889c967b1a2-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-sm5wr\" (UID: \"98fa1395-052d-4cb6-a5cc-e889c967b1a2\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr"
Sep 19 10:35:55.690814 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.690796    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56fk9\" (UniqueName: \"kubernetes.io/projected/98fa1395-052d-4cb6-a5cc-e889c967b1a2-kube-api-access-56fk9\") pod \"monitoring-plugin-5984459ff8-sm5wr\" (UID: \"98fa1395-052d-4cb6-a5cc-e889c967b1a2\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr"
Sep 19 10:35:55.768789 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.768738    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr"
Sep 19 10:35:55.845174 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.845149    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-69b884cc84-bmrcv"]
Sep 19 10:35:55.847444 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.847428    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:55.850161 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.850139    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 19 10:35:55.850331 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.850289    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 19 10:35:55.850393 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.850332    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 19 10:35:55.850993 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.850975    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 19 10:35:55.851091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.851052    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 19 10:35:55.851304 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.851289    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2p8x4\""
Sep 19 10:35:55.857155 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.857137    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 19 10:35:55.866866 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.866845    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69b884cc84-bmrcv"]
Sep 19 10:35:55.980946 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.980911    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f8a20344-2487-4643-856f-67df241098bb-telemeter-client-tls\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:55.981089 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.980963    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f8a20344-2487-4643-856f-67df241098bb-federate-client-tls\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:55.981089 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.981014    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qbtc\" (UniqueName: \"kubernetes.io/projected/f8a20344-2487-4643-856f-67df241098bb-kube-api-access-6qbtc\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:55.981089 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.981048    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8a20344-2487-4643-856f-67df241098bb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:55.981089 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.981082    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a20344-2487-4643-856f-67df241098bb-serving-certs-ca-bundle\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:55.981303 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.981105    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a20344-2487-4643-856f-67df241098bb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:55.981303 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.981138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f8a20344-2487-4643-856f-67df241098bb-secret-telemeter-client\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:55.981303 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:55.981158    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a20344-2487-4643-856f-67df241098bb-metrics-client-ca\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.041671 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.041645    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr"]
Sep 19 10:35:56.043584 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:56.043553    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98fa1395_052d_4cb6_a5cc_e889c967b1a2.slice/crio-a54c506de5aa98dfee88fa0bc20d71b7fa1f45b180dcb463b1e9a7314e88b284 WatchSource:0}: Error finding container a54c506de5aa98dfee88fa0bc20d71b7fa1f45b180dcb463b1e9a7314e88b284: Status 404 returned error can't find the container with id a54c506de5aa98dfee88fa0bc20d71b7fa1f45b180dcb463b1e9a7314e88b284
Sep 19 10:35:56.082256 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.082235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qbtc\" (UniqueName: \"kubernetes.io/projected/f8a20344-2487-4643-856f-67df241098bb-kube-api-access-6qbtc\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.082381 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.082272    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8a20344-2487-4643-856f-67df241098bb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.082381 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.082298    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a20344-2487-4643-856f-67df241098bb-serving-certs-ca-bundle\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.082381 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.082323    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a20344-2487-4643-856f-67df241098bb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.082381 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.082355    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f8a20344-2487-4643-856f-67df241098bb-secret-telemeter-client\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.082635 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.082380    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a20344-2487-4643-856f-67df241098bb-metrics-client-ca\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.082635 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.082417    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f8a20344-2487-4643-856f-67df241098bb-telemeter-client-tls\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.082635 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.082451    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f8a20344-2487-4643-856f-67df241098bb-federate-client-tls\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.083345 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.083300    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a20344-2487-4643-856f-67df241098bb-serving-certs-ca-bundle\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.083739 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.083718    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8a20344-2487-4643-856f-67df241098bb-metrics-client-ca\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.083837 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.083807    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8a20344-2487-4643-856f-67df241098bb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.085537 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.085503    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f8a20344-2487-4643-856f-67df241098bb-federate-client-tls\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.085657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.085589    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f8a20344-2487-4643-856f-67df241098bb-secret-telemeter-client\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.086164 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.086138    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8a20344-2487-4643-856f-67df241098bb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.086164 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.086157    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f8a20344-2487-4643-856f-67df241098bb-telemeter-client-tls\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.091394 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.091374    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qbtc\" (UniqueName: \"kubernetes.io/projected/f8a20344-2487-4643-856f-67df241098bb-kube-api-access-6qbtc\") pod \"telemeter-client-69b884cc84-bmrcv\" (UID: \"f8a20344-2487-4643-856f-67df241098bb\") " pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.157536 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.157484    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv"
Sep 19 10:35:56.440483 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.440362    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69b884cc84-bmrcv"]
Sep 19 10:35:56.440483 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.440443    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr" event={"ID":"98fa1395-052d-4cb6-a5cc-e889c967b1a2","Type":"ContainerStarted","Data":"a54c506de5aa98dfee88fa0bc20d71b7fa1f45b180dcb463b1e9a7314e88b284"}
Sep 19 10:35:56.441984 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.441943    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n" event={"ID":"47c4dd98-e96a-426c-a40b-821612719f6a","Type":"ContainerStarted","Data":"d9a502a578c821536af0ad94d286969fa39f84c2699edf9d19863ad35053cf7f"}
Sep 19 10:35:56.775154 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:56.775069    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:56.780326 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:35:56.780298    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a20344_2487_4643_856f_67df241098bb.slice/crio-0ce9a07f981764a10b7addd17d1fa33b08a901b5ea2601b770de4fc097b907d2 WatchSource:0}: Error finding container 0ce9a07f981764a10b7addd17d1fa33b08a901b5ea2601b770de4fc097b907d2: Status 404 returned error can't find the container with id 0ce9a07f981764a10b7addd17d1fa33b08a901b5ea2601b770de4fc097b907d2
Sep 19 10:35:57.448896 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:57.448853    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv" event={"ID":"f8a20344-2487-4643-856f-67df241098bb","Type":"ContainerStarted","Data":"0ce9a07f981764a10b7addd17d1fa33b08a901b5ea2601b770de4fc097b907d2"}
Sep 19 10:35:57.450983 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:57.450952    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n" event={"ID":"47c4dd98-e96a-426c-a40b-821612719f6a","Type":"ContainerStarted","Data":"f3f8c6249c156ff724ef7ee6fa897cab1f9de5105295ab459488054baba05269"}
Sep 19 10:35:57.474570 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:57.474504    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n" podStartSLOduration=2.146973138 podStartE2EDuration="3.474484939s" totalImagesPullingTime="1.327511801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:35:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:57.473503924 +0000 UTC m=+78.385197443" watchObservedRunningTime="2026-09-19 10:35:57.474484939 +0000 UTC m=+78.386178459"
Sep 19 10:35:57.918230 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:57.918198    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:58.022317 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:58.022214    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:58.457691 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:58.457658    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr" event={"ID":"98fa1395-052d-4cb6-a5cc-e889c967b1a2","Type":"ContainerStarted","Data":"3019d024353fc7b0928cae9f9366d3cfa9cd97dce88c5e975eb397c09bb6e16c"}
Sep 19 10:35:58.479667 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:58.479605    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr" podStartSLOduration=1.780960419 podStartE2EDuration="3.479588034s" totalImagesPullingTime="1.698627615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:35:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:35:58.47876395 +0000 UTC m=+79.390457472" watchObservedRunningTime="2026-09-19 10:35:58.479588034 +0000 UTC m=+79.391281552"
Sep 19 10:35:59.242792 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:59.242736    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:59.306565 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:59.306518    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:59.361775 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:59.361753    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56b557bc79-sqmkg"
Sep 19 10:35:59.464021 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:59.463971    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv" event={"ID":"f8a20344-2487-4643-856f-67df241098bb","Type":"ContainerStarted","Data":"64563bcfea9326de383dfaa6598433dda727c607942e0f3b577ced2187e65b99"}
Sep 19 10:35:59.464393 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:59.464083    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr"
Sep 19 10:35:59.469236 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:59.469212    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-sm5wr"
Sep 19 10:35:59.503197 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:59.503035    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:59.585415 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:59.585337    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:59.686893 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:59.686869    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:35:59.925216 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:35:59.925187    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:00.039888 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:00.039795    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:00.469865 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:00.469804    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv" event={"ID":"f8a20344-2487-4643-856f-67df241098bb","Type":"ContainerStarted","Data":"d822e0455493192cc6ec1a06308c9d92f567ab8e5f5d3b937b65063da61e6d71"}
Sep 19 10:36:00.470327 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:00.469880    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv" event={"ID":"f8a20344-2487-4643-856f-67df241098bb","Type":"ContainerStarted","Data":"07b90a7c9ce4010fb79c4db80972102d4821975e30d1063be01444a49d56dd34"}
Sep 19 10:36:00.496239 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:00.496197    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-69b884cc84-bmrcv" podStartSLOduration=2.684595292 podStartE2EDuration="5.496177815s" totalImagesPullingTime="2.811582523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:35:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:36:00.494302728 +0000 UTC m=+81.405996249" watchObservedRunningTime="2026-09-19 10:36:00.496177815 +0000 UTC m=+81.407871335"
Sep 19 10:36:05.070700 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.070666    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:05.129318 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.129292    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:05.529287 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.529256    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-xjb6l"]
Sep 19 10:36:05.532282 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.532263    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-xjb6l"
Sep 19 10:36:05.534779 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.534755    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 19 10:36:05.534896 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.534881    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7rx2r\""
Sep 19 10:36:05.534951 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.534929    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 19 10:36:05.545956 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.545932    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:05.546096 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.546080    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-xjb6l"]
Sep 19 10:36:05.642891 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.642863    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:05.664494 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.664476    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cctr\" (UniqueName: \"kubernetes.io/projected/90b5fc61-0581-47ff-a61b-31cd617c1352-kube-api-access-8cctr\") pod \"downloads-6594f4c75-xjb6l\" (UID: \"90b5fc61-0581-47ff-a61b-31cd617c1352\") " pod="openshift-console/downloads-6594f4c75-xjb6l"
Sep 19 10:36:05.704855 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.704828    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:05.765064 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.765037    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cctr\" (UniqueName: \"kubernetes.io/projected/90b5fc61-0581-47ff-a61b-31cd617c1352-kube-api-access-8cctr\") pod \"downloads-6594f4c75-xjb6l\" (UID: \"90b5fc61-0581-47ff-a61b-31cd617c1352\") " pod="openshift-console/downloads-6594f4c75-xjb6l"
Sep 19 10:36:05.773198 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.773175    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cctr\" (UniqueName: \"kubernetes.io/projected/90b5fc61-0581-47ff-a61b-31cd617c1352-kube-api-access-8cctr\") pod \"downloads-6594f4c75-xjb6l\" (UID: \"90b5fc61-0581-47ff-a61b-31cd617c1352\") " pod="openshift-console/downloads-6594f4c75-xjb6l"
Sep 19 10:36:05.826006 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.825936    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:05.842008 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:05.841972    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-xjb6l"
Sep 19 10:36:06.109785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:06.109764    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-xjb6l"]
Sep 19 10:36:06.112101 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:36:06.112073    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b5fc61_0581_47ff_a61b_31cd617c1352.slice/crio-ae6b21cfe6255fe301e25433e8827073d460cfbd5368bde1902109a611f26f40 WatchSource:0}: Error finding container ae6b21cfe6255fe301e25433e8827073d460cfbd5368bde1902109a611f26f40: Status 404 returned error can't find the container with id ae6b21cfe6255fe301e25433e8827073d460cfbd5368bde1902109a611f26f40
Sep 19 10:36:06.275918 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:06.275889    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:06.378767 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:06.378704    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:06.497681 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:06.497642    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-xjb6l" event={"ID":"90b5fc61-0581-47ff-a61b-31cd617c1352","Type":"ContainerStarted","Data":"ae6b21cfe6255fe301e25433e8827073d460cfbd5368bde1902109a611f26f40"}
Sep 19 10:36:06.600543 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:06.600489    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:06.669616 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:06.669517    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:07.007617 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:07.007488    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:07.235577 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:07.235545    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:07.413290 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:07.413203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:07.628193 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:07.628159    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:08.436358 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:08.436324    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:08.534121 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:08.534086    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:12.117625 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:12.117590    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:12.172452 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:12.172422    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:15.197047 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:15.197014    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:36:15.197463 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:15.197076    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:36:20.060736 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:20.060697    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:20.118918 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:20.118869    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:20.638356 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:20.638320    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:20.703609 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:20.703580    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:23.427206 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:23.427175    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9f2kq"
Sep 19 10:36:23.570614 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:23.570575    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-xjb6l" event={"ID":"90b5fc61-0581-47ff-a61b-31cd617c1352","Type":"ContainerStarted","Data":"735418cf34be8ecf26f145e82120e0a0d715fe67773f3340394ffea4c6530575"}
Sep 19 10:36:23.570894 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:23.570871    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-xjb6l"
Sep 19 10:36:23.585613 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:23.585590    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-xjb6l"
Sep 19 10:36:23.607251 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:23.607198    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-xjb6l" podStartSLOduration=1.699169936 podStartE2EDuration="18.607177067s" totalImagesPullingTime="16.908007131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:36:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:36:23.587713929 +0000 UTC m=+104.499407448" watchObservedRunningTime="2026-09-19 10:36:23.607177067 +0000 UTC m=+104.518870587"
Sep 19 10:36:24.234444 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:24.234409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:25.027422 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.027386    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c4f54bb7f-np62b"]
Sep 19 10:36:25.057435 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.057410    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c4f54bb7f-np62b"]
Sep 19 10:36:25.057606 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.057507    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.060486 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.060457    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 19 10:36:25.061435 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.061350    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 19 10:36:25.061435 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.061396    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z4n2v\""
Sep 19 10:36:25.061659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.061443    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 19 10:36:25.061659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.061358    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 19 10:36:25.061763 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.061709    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 19 10:36:25.066478 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.066436    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 19 10:36:25.228228 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.228190    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78crl\" (UniqueName: \"kubernetes.io/projected/51217232-32f8-4984-a147-ad0c8ed8d4eb-kube-api-access-78crl\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.228417 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.228264    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-oauth-serving-cert\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.228417 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.228301    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-serving-cert\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.228417 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.228355    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-oauth-config\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.228607 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.228459    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-service-ca\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.228607 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.228507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-config\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.228607 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.228557    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-trusted-ca-bundle\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.329624 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.329551    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-oauth-serving-cert\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.329624 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.329603    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-serving-cert\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.329864 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.329636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-oauth-config\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.329864 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.329676    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-service-ca\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.329864 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.329708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-config\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.329864 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.329740    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-trusted-ca-bundle\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.329864 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.329789    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78crl\" (UniqueName: \"kubernetes.io/projected/51217232-32f8-4984-a147-ad0c8ed8d4eb-kube-api-access-78crl\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.330385 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.330358    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-oauth-serving-cert\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.330570 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.330542    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-trusted-ca-bundle\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.330570 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.330399    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-service-ca\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.331109 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.331084    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-config\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.332446 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.332418    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-oauth-config\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.332584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.332559    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-serving-cert\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.338574 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.338554    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78crl\" (UniqueName: \"kubernetes.io/projected/51217232-32f8-4984-a147-ad0c8ed8d4eb-kube-api-access-78crl\") pod \"console-5c4f54bb7f-np62b\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") " pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.369098 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.369073    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:25.673865 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:25.673781    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c4f54bb7f-np62b"]
Sep 19 10:36:25.676831 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:36:25.676807    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51217232_32f8_4984_a147_ad0c8ed8d4eb.slice/crio-c68dad420a8860ff420fcc5df53f6248dcdd39576e48f5c30cb889dd86481e8a WatchSource:0}: Error finding container c68dad420a8860ff420fcc5df53f6248dcdd39576e48f5c30cb889dd86481e8a: Status 404 returned error can't find the container with id c68dad420a8860ff420fcc5df53f6248dcdd39576e48f5c30cb889dd86481e8a
Sep 19 10:36:26.588913 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:26.588869    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4f54bb7f-np62b" event={"ID":"51217232-32f8-4984-a147-ad0c8ed8d4eb","Type":"ContainerStarted","Data":"c68dad420a8860ff420fcc5df53f6248dcdd39576e48f5c30cb889dd86481e8a"}
Sep 19 10:36:27.681173 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:27.681090    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:27.782269 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:27.782173    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:27.847512 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:27.847448    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:30.042240 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:30.042203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:30.098038 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:30.097982    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:30.839483 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:30.839434    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:30.894476 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:30.894440    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:31.614570 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:31.614471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4f54bb7f-np62b" event={"ID":"51217232-32f8-4984-a147-ad0c8ed8d4eb","Type":"ContainerStarted","Data":"10d6d0e20a04cc87b8541f14dc698da07fea4033e73fbe1ecaaefc3c250ddc45"}
Sep 19 10:36:31.633840 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:31.633796    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c4f54bb7f-np62b" podStartSLOduration=0.994047054 podStartE2EDuration="6.633780333s" totalImagesPullingTime="5.639733279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:36:31.6321559 +0000 UTC m=+112.543849421" watchObservedRunningTime="2026-09-19 10:36:31.633780333 +0000 UTC m=+112.545473850"
Sep 19 10:36:35.068656 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:35.068626    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:35.132156 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:35.132128    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:35.201830 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:35.201804    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:36:35.205620 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:35.205599    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7674648dc8-9hb4n"
Sep 19 10:36:35.369247 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:35.369173    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:35.369247 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:35.369225    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:35.373868 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:35.373846    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:35.633145 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:35.633083    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:36:35.775801 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:35.775766    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:35.834214 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:35.834182    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:35.905398 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:35.905331    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:50.066455 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:50.066415    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:36:50.121271 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:36:50.121245    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:37:06.660320 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:37:06.660284    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:37:06.722442 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:37:06.722415    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:37:18.616276 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:37:18.616245    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:37:18.683417 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:37:18.683388    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:37:24.109570 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:37:24.109470    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:37:24.167059 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:37:24.167029    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:37:35.020683 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:37:35.020646    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c4f54bb7f-np62b"]
Sep 19 10:37:35.080091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:37:35.080063    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:37:35.182383 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:37:35.182351    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:37:35.282863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:37:35.282799    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:37:49.593689 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:37:49.593652    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:00.039743 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.039679    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c4f54bb7f-np62b" podUID="51217232-32f8-4984-a147-ad0c8ed8d4eb" containerName="console" containerID="cri-o://10d6d0e20a04cc87b8541f14dc698da07fea4033e73fbe1ecaaefc3c250ddc45" gracePeriod=15
Sep 19 10:38:00.272134 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.272114    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:38:00.342828 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.342756    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-serving-cert\") pod \"51217232-32f8-4984-a147-ad0c8ed8d4eb\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") "
Sep 19 10:38:00.342828 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.342805    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-oauth-serving-cert\") pod \"51217232-32f8-4984-a147-ad0c8ed8d4eb\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") "
Sep 19 10:38:00.343007 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.342876    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-oauth-config\") pod \"51217232-32f8-4984-a147-ad0c8ed8d4eb\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") "
Sep 19 10:38:00.343007 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.342904    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78crl\" (UniqueName: \"kubernetes.io/projected/51217232-32f8-4984-a147-ad0c8ed8d4eb-kube-api-access-78crl\") pod \"51217232-32f8-4984-a147-ad0c8ed8d4eb\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") "
Sep 19 10:38:00.343007 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.342929    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-service-ca\") pod \"51217232-32f8-4984-a147-ad0c8ed8d4eb\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") "
Sep 19 10:38:00.343007 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.342951    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-trusted-ca-bundle\") pod \"51217232-32f8-4984-a147-ad0c8ed8d4eb\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") "
Sep 19 10:38:00.343007 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.342980    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-config\") pod \"51217232-32f8-4984-a147-ad0c8ed8d4eb\" (UID: \"51217232-32f8-4984-a147-ad0c8ed8d4eb\") "
Sep 19 10:38:00.343293 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.343257    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "51217232-32f8-4984-a147-ad0c8ed8d4eb" (UID: "51217232-32f8-4984-a147-ad0c8ed8d4eb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:38:00.343395 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.343371    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "51217232-32f8-4984-a147-ad0c8ed8d4eb" (UID: "51217232-32f8-4984-a147-ad0c8ed8d4eb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:38:00.343395 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.343382    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-service-ca" (OuterVolumeSpecName: "service-ca") pod "51217232-32f8-4984-a147-ad0c8ed8d4eb" (UID: "51217232-32f8-4984-a147-ad0c8ed8d4eb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:38:00.343503 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.343461    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-config" (OuterVolumeSpecName: "console-config") pod "51217232-32f8-4984-a147-ad0c8ed8d4eb" (UID: "51217232-32f8-4984-a147-ad0c8ed8d4eb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:38:00.344960 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.344941    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "51217232-32f8-4984-a147-ad0c8ed8d4eb" (UID: "51217232-32f8-4984-a147-ad0c8ed8d4eb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:38:00.345036 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.345003    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "51217232-32f8-4984-a147-ad0c8ed8d4eb" (UID: "51217232-32f8-4984-a147-ad0c8ed8d4eb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:38:00.345115 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.345097    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51217232-32f8-4984-a147-ad0c8ed8d4eb-kube-api-access-78crl" (OuterVolumeSpecName: "kube-api-access-78crl") pod "51217232-32f8-4984-a147-ad0c8ed8d4eb" (UID: "51217232-32f8-4984-a147-ad0c8ed8d4eb"). InnerVolumeSpecName "kube-api-access-78crl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:38:00.444109 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.444083    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-oauth-serving-cert\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:38:00.444109 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.444106    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-oauth-config\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:38:00.444274 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.444115    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78crl\" (UniqueName: \"kubernetes.io/projected/51217232-32f8-4984-a147-ad0c8ed8d4eb-kube-api-access-78crl\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:38:00.444274 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.444126    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:38:00.444274 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.444134    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-trusted-ca-bundle\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:38:00.444274 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.444142    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-config\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:38:00.444274 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.444150    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51217232-32f8-4984-a147-ad0c8ed8d4eb-console-serving-cert\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:38:00.928224 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.928189    2579 generic.go:356] "Generic (PLEG): container finished" podID="51217232-32f8-4984-a147-ad0c8ed8d4eb" containerID="10d6d0e20a04cc87b8541f14dc698da07fea4033e73fbe1ecaaefc3c250ddc45" exitCode=0
Sep 19 10:38:00.928400 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.928230    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c4f54bb7f-np62b"
Sep 19 10:38:00.928400 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.928252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4f54bb7f-np62b" event={"ID":"51217232-32f8-4984-a147-ad0c8ed8d4eb","Type":"ContainerDied","Data":"10d6d0e20a04cc87b8541f14dc698da07fea4033e73fbe1ecaaefc3c250ddc45"}
Sep 19 10:38:00.928400 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.928290    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c4f54bb7f-np62b" event={"ID":"51217232-32f8-4984-a147-ad0c8ed8d4eb","Type":"ContainerDied","Data":"c68dad420a8860ff420fcc5df53f6248dcdd39576e48f5c30cb889dd86481e8a"}
Sep 19 10:38:00.928400 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.928309    2579 scope.go:117] "RemoveContainer" containerID="10d6d0e20a04cc87b8541f14dc698da07fea4033e73fbe1ecaaefc3c250ddc45"
Sep 19 10:38:00.936715 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.936699    2579 scope.go:117] "RemoveContainer" containerID="10d6d0e20a04cc87b8541f14dc698da07fea4033e73fbe1ecaaefc3c250ddc45"
Sep 19 10:38:00.936966 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:38:00.936949    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10d6d0e20a04cc87b8541f14dc698da07fea4033e73fbe1ecaaefc3c250ddc45\": container with ID starting with 10d6d0e20a04cc87b8541f14dc698da07fea4033e73fbe1ecaaefc3c250ddc45 not found: ID does not exist" containerID="10d6d0e20a04cc87b8541f14dc698da07fea4033e73fbe1ecaaefc3c250ddc45"
Sep 19 10:38:00.937017 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.936976    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10d6d0e20a04cc87b8541f14dc698da07fea4033e73fbe1ecaaefc3c250ddc45"} err="failed to get container status \"10d6d0e20a04cc87b8541f14dc698da07fea4033e73fbe1ecaaefc3c250ddc45\": rpc error: code = NotFound desc = could not find container \"10d6d0e20a04cc87b8541f14dc698da07fea4033e73fbe1ecaaefc3c250ddc45\": container with ID starting with 10d6d0e20a04cc87b8541f14dc698da07fea4033e73fbe1ecaaefc3c250ddc45 not found: ID does not exist"
Sep 19 10:38:00.948831 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.948810    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c4f54bb7f-np62b"]
Sep 19 10:38:00.952505 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:00.952483    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c4f54bb7f-np62b"]
Sep 19 10:38:01.113783 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:01.113744    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51217232-32f8-4984-a147-ad0c8ed8d4eb" path="/var/lib/kubelet/pods/51217232-32f8-4984-a147-ad0c8ed8d4eb/volumes"
Sep 19 10:38:01.836427 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:01.836394    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:01.894090 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:01.894058    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:16.519747 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:16.519711    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:16.596309 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:16.596277    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:17.277410 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.277381    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"]
Sep 19 10:38:17.277665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.277653    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51217232-32f8-4984-a147-ad0c8ed8d4eb" containerName="console"
Sep 19 10:38:17.277715 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.277666    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="51217232-32f8-4984-a147-ad0c8ed8d4eb" containerName="console"
Sep 19 10:38:17.277748 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.277717    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="51217232-32f8-4984-a147-ad0c8ed8d4eb" containerName="console"
Sep 19 10:38:17.280426 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.280410    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"
Sep 19 10:38:17.283033 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.283010    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rk9vp\""
Sep 19 10:38:17.283193 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.283179    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 10:38:17.284201 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.284180    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 10:38:17.292904 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.292883    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"]
Sep 19 10:38:17.365674 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.365644    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efb5c44f-95a6-43e7-8dba-e902fd223ba1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s\" (UID: \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"
Sep 19 10:38:17.365801 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.365687    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efb5c44f-95a6-43e7-8dba-e902fd223ba1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s\" (UID: \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"
Sep 19 10:38:17.365801 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.365728    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5vw7\" (UniqueName: \"kubernetes.io/projected/efb5c44f-95a6-43e7-8dba-e902fd223ba1-kube-api-access-z5vw7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s\" (UID: \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"
Sep 19 10:38:17.466568 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.466542    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5vw7\" (UniqueName: \"kubernetes.io/projected/efb5c44f-95a6-43e7-8dba-e902fd223ba1-kube-api-access-z5vw7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s\" (UID: \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"
Sep 19 10:38:17.466701 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.466589    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efb5c44f-95a6-43e7-8dba-e902fd223ba1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s\" (UID: \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"
Sep 19 10:38:17.466701 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.466625    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efb5c44f-95a6-43e7-8dba-e902fd223ba1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s\" (UID: \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"
Sep 19 10:38:17.467020 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.467000    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efb5c44f-95a6-43e7-8dba-e902fd223ba1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s\" (UID: \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"
Sep 19 10:38:17.467020 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.467015    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efb5c44f-95a6-43e7-8dba-e902fd223ba1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s\" (UID: \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"
Sep 19 10:38:17.475218 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.475199    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5vw7\" (UniqueName: \"kubernetes.io/projected/efb5c44f-95a6-43e7-8dba-e902fd223ba1-kube-api-access-z5vw7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s\" (UID: \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"
Sep 19 10:38:17.589460 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.589396    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"
Sep 19 10:38:17.860905 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.860818    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"]
Sep 19 10:38:17.863402 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:38:17.863364    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb5c44f_95a6_43e7_8dba_e902fd223ba1.slice/crio-e95523c69d10d23c1b7770444a5eb2ae7873204eca4f2a0b0994cc24be4817b4 WatchSource:0}: Error finding container e95523c69d10d23c1b7770444a5eb2ae7873204eca4f2a0b0994cc24be4817b4: Status 404 returned error can't find the container with id e95523c69d10d23c1b7770444a5eb2ae7873204eca4f2a0b0994cc24be4817b4
Sep 19 10:38:17.988056 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:17.988025    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s" event={"ID":"efb5c44f-95a6-43e7-8dba-e902fd223ba1","Type":"ContainerStarted","Data":"e95523c69d10d23c1b7770444a5eb2ae7873204eca4f2a0b0994cc24be4817b4"}
Sep 19 10:38:24.011741 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:24.011652    2579 generic.go:356] "Generic (PLEG): container finished" podID="efb5c44f-95a6-43e7-8dba-e902fd223ba1" containerID="0a24fdf823f26ac32b01cf8211ffe96accf9d5cf0e177bddfed19a24b7a378db" exitCode=0
Sep 19 10:38:24.012117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:24.011736    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s" event={"ID":"efb5c44f-95a6-43e7-8dba-e902fd223ba1","Type":"ContainerDied","Data":"0a24fdf823f26ac32b01cf8211ffe96accf9d5cf0e177bddfed19a24b7a378db"}
Sep 19 10:38:26.020893 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:26.020861    2579 generic.go:356] "Generic (PLEG): container finished" podID="efb5c44f-95a6-43e7-8dba-e902fd223ba1" containerID="d0d7dd05032d24a966b4fa9b066558089572300c09211a6dc55d69c7dea7aaa2" exitCode=0
Sep 19 10:38:26.021278 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:26.020911    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s" event={"ID":"efb5c44f-95a6-43e7-8dba-e902fd223ba1","Type":"ContainerDied","Data":"d0d7dd05032d24a966b4fa9b066558089572300c09211a6dc55d69c7dea7aaa2"}
Sep 19 10:38:33.056233 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:33.056142    2579 generic.go:356] "Generic (PLEG): container finished" podID="efb5c44f-95a6-43e7-8dba-e902fd223ba1" containerID="0bbd274ac4f7b6c6f4c082a9fa15c6ce16bd9ddb77f3ef54a09a3223a7ca6689" exitCode=0
Sep 19 10:38:33.056566 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:33.056232    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s" event={"ID":"efb5c44f-95a6-43e7-8dba-e902fd223ba1","Type":"ContainerDied","Data":"0bbd274ac4f7b6c6f4c082a9fa15c6ce16bd9ddb77f3ef54a09a3223a7ca6689"}
Sep 19 10:38:34.174953 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:34.174931    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"
Sep 19 10:38:34.310835 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:34.310765    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5vw7\" (UniqueName: \"kubernetes.io/projected/efb5c44f-95a6-43e7-8dba-e902fd223ba1-kube-api-access-z5vw7\") pod \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\" (UID: \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\") "
Sep 19 10:38:34.310835 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:34.310821    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efb5c44f-95a6-43e7-8dba-e902fd223ba1-bundle\") pod \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\" (UID: \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\") "
Sep 19 10:38:34.310989 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:34.310876    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efb5c44f-95a6-43e7-8dba-e902fd223ba1-util\") pod \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\" (UID: \"efb5c44f-95a6-43e7-8dba-e902fd223ba1\") "
Sep 19 10:38:34.311660 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:34.311621    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb5c44f-95a6-43e7-8dba-e902fd223ba1-bundle" (OuterVolumeSpecName: "bundle") pod "efb5c44f-95a6-43e7-8dba-e902fd223ba1" (UID: "efb5c44f-95a6-43e7-8dba-e902fd223ba1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:38:34.312882 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:34.312849    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efb5c44f-95a6-43e7-8dba-e902fd223ba1-kube-api-access-z5vw7" (OuterVolumeSpecName: "kube-api-access-z5vw7") pod "efb5c44f-95a6-43e7-8dba-e902fd223ba1" (UID: "efb5c44f-95a6-43e7-8dba-e902fd223ba1"). InnerVolumeSpecName "kube-api-access-z5vw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:38:34.315753 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:34.315731    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb5c44f-95a6-43e7-8dba-e902fd223ba1-util" (OuterVolumeSpecName: "util") pod "efb5c44f-95a6-43e7-8dba-e902fd223ba1" (UID: "efb5c44f-95a6-43e7-8dba-e902fd223ba1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:38:34.411821 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:34.411797    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5vw7\" (UniqueName: \"kubernetes.io/projected/efb5c44f-95a6-43e7-8dba-e902fd223ba1-kube-api-access-z5vw7\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:38:34.411821 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:34.411818    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/efb5c44f-95a6-43e7-8dba-e902fd223ba1-bundle\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:38:34.411956 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:34.411827    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/efb5c44f-95a6-43e7-8dba-e902fd223ba1-util\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:38:35.065251 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:35.065224    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s"
Sep 19 10:38:35.065251 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:35.065229    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540krs5s" event={"ID":"efb5c44f-95a6-43e7-8dba-e902fd223ba1","Type":"ContainerDied","Data":"e95523c69d10d23c1b7770444a5eb2ae7873204eca4f2a0b0994cc24be4817b4"}
Sep 19 10:38:35.065437 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:35.065292    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e95523c69d10d23c1b7770444a5eb2ae7873204eca4f2a0b0994cc24be4817b4"
Sep 19 10:38:38.553711 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:38.553677    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:39.904007 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:39.903973    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:40.422873 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.422844    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ztdk9"]
Sep 19 10:38:40.423100 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.423089    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb5c44f-95a6-43e7-8dba-e902fd223ba1" containerName="util"
Sep 19 10:38:40.423146 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.423102    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb5c44f-95a6-43e7-8dba-e902fd223ba1" containerName="util"
Sep 19 10:38:40.423146 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.423113    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb5c44f-95a6-43e7-8dba-e902fd223ba1" containerName="pull"
Sep 19 10:38:40.423146 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.423119    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb5c44f-95a6-43e7-8dba-e902fd223ba1" containerName="pull"
Sep 19 10:38:40.423146 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.423136    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb5c44f-95a6-43e7-8dba-e902fd223ba1" containerName="extract"
Sep 19 10:38:40.423146 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.423142    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb5c44f-95a6-43e7-8dba-e902fd223ba1" containerName="extract"
Sep 19 10:38:40.423356 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.423195    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="efb5c44f-95a6-43e7-8dba-e902fd223ba1" containerName="extract"
Sep 19 10:38:40.470717 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.470690    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ztdk9"]
Sep 19 10:38:40.470852 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.470778    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ztdk9"
Sep 19 10:38:40.473495 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.473475    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-ljsvt\""
Sep 19 10:38:40.473633 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.473476    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 19 10:38:40.473977 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.473958    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 19 10:38:40.553580 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.553556    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6be55842-5c75-4df1-8b5e-f5c3afd967f4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ztdk9\" (UID: \"6be55842-5c75-4df1-8b5e-f5c3afd967f4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ztdk9"
Sep 19 10:38:40.553696 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.553598    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj9sc\" (UniqueName: \"kubernetes.io/projected/6be55842-5c75-4df1-8b5e-f5c3afd967f4-kube-api-access-vj9sc\") pod \"cert-manager-operator-controller-manager-55c444bf86-ztdk9\" (UID: \"6be55842-5c75-4df1-8b5e-f5c3afd967f4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ztdk9"
Sep 19 10:38:40.654224 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.654199    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6be55842-5c75-4df1-8b5e-f5c3afd967f4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ztdk9\" (UID: \"6be55842-5c75-4df1-8b5e-f5c3afd967f4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ztdk9"
Sep 19 10:38:40.654307 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.654239    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj9sc\" (UniqueName: \"kubernetes.io/projected/6be55842-5c75-4df1-8b5e-f5c3afd967f4-kube-api-access-vj9sc\") pod \"cert-manager-operator-controller-manager-55c444bf86-ztdk9\" (UID: \"6be55842-5c75-4df1-8b5e-f5c3afd967f4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ztdk9"
Sep 19 10:38:40.654576 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.654559    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6be55842-5c75-4df1-8b5e-f5c3afd967f4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ztdk9\" (UID: \"6be55842-5c75-4df1-8b5e-f5c3afd967f4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ztdk9"
Sep 19 10:38:40.666055 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.666034    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj9sc\" (UniqueName: \"kubernetes.io/projected/6be55842-5c75-4df1-8b5e-f5c3afd967f4-kube-api-access-vj9sc\") pod \"cert-manager-operator-controller-manager-55c444bf86-ztdk9\" (UID: \"6be55842-5c75-4df1-8b5e-f5c3afd967f4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ztdk9"
Sep 19 10:38:40.780154 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:40.780096    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ztdk9"
Sep 19 10:38:41.068923 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:41.068901    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ztdk9"]
Sep 19 10:38:41.071188 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:38:41.071166    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6be55842_5c75_4df1_8b5e_f5c3afd967f4.slice/crio-33b90862936017b4987d928d27066d31217a0a42153ef9007a42877c86777b3e WatchSource:0}: Error finding container 33b90862936017b4987d928d27066d31217a0a42153ef9007a42877c86777b3e: Status 404 returned error can't find the container with id 33b90862936017b4987d928d27066d31217a0a42153ef9007a42877c86777b3e
Sep 19 10:38:41.091759 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:41.091727    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ztdk9" event={"ID":"6be55842-5c75-4df1-8b5e-f5c3afd967f4","Type":"ContainerStarted","Data":"33b90862936017b4987d928d27066d31217a0a42153ef9007a42877c86777b3e"}
Sep 19 10:38:41.812177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:41.812141    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:42.125211 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:42.125127    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:42.248512 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:42.248479    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:43.101268 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:43.101232    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ztdk9" event={"ID":"6be55842-5c75-4df1-8b5e-f5c3afd967f4","Type":"ContainerStarted","Data":"e75c3c2a9f2a63f972db38eb185e6822556ec99a08f2ac342389d9b481bbd0cf"}
Sep 19 10:38:43.125418 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:43.125373    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ztdk9" podStartSLOduration=1.770606348 podStartE2EDuration="3.125360603s" totalImagesPullingTime="1.354754255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:38:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:38:43.124081426 +0000 UTC m=+244.035774944" watchObservedRunningTime="2026-09-19 10:38:43.125360603 +0000 UTC m=+244.037054121"
Sep 19 10:38:43.461995 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:43.461966    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:43.567699 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:43.567670    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:44.059238 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.059205    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:44.174880 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.174849    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:44.397555 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.397472    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"]
Sep 19 10:38:44.399862 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.399846    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"
Sep 19 10:38:44.402355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.402332    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 10:38:44.402473 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.402396    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 10:38:44.402473 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.402462    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rk9vp\""
Sep 19 10:38:44.415316 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.415295    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"]
Sep 19 10:38:44.484572 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.484540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg\" (UID: \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"
Sep 19 10:38:44.484688 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.484590    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbnx6\" (UniqueName: \"kubernetes.io/projected/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-kube-api-access-xbnx6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg\" (UID: \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"
Sep 19 10:38:44.484688 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.484614    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg\" (UID: \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"
Sep 19 10:38:44.585356 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.585328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg\" (UID: \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"
Sep 19 10:38:44.585543 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.585367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbnx6\" (UniqueName: \"kubernetes.io/projected/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-kube-api-access-xbnx6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg\" (UID: \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"
Sep 19 10:38:44.585543 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.585387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg\" (UID: \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"
Sep 19 10:38:44.585775 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.585756    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg\" (UID: \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"
Sep 19 10:38:44.585834 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.585801    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg\" (UID: \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"
Sep 19 10:38:44.594904 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.594884    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbnx6\" (UniqueName: \"kubernetes.io/projected/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-kube-api-access-xbnx6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg\" (UID: \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"
Sep 19 10:38:44.708261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.708233    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"
Sep 19 10:38:44.974717 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:44.974688    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"]
Sep 19 10:38:44.976838 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:38:44.976816    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod902a37e7_4fae_4f82_ab3b_df723e1e9dd5.slice/crio-eb5505907b6806de29689af4eb46e8217228f80729d4ab29e116dbb46460dca3 WatchSource:0}: Error finding container eb5505907b6806de29689af4eb46e8217228f80729d4ab29e116dbb46460dca3: Status 404 returned error can't find the container with id eb5505907b6806de29689af4eb46e8217228f80729d4ab29e116dbb46460dca3
Sep 19 10:38:45.110494 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:45.110463    2579 generic.go:356] "Generic (PLEG): container finished" podID="902a37e7-4fae-4f82-ab3b-df723e1e9dd5" containerID="4dce92e7fefc5479c69472acb5b89c6606e7546f2b7b4adb77821de99c9c13a6" exitCode=0
Sep 19 10:38:45.114251 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:45.114231    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg" event={"ID":"902a37e7-4fae-4f82-ab3b-df723e1e9dd5","Type":"ContainerDied","Data":"4dce92e7fefc5479c69472acb5b89c6606e7546f2b7b4adb77821de99c9c13a6"}
Sep 19 10:38:45.114317 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:45.114266    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg" event={"ID":"902a37e7-4fae-4f82-ab3b-df723e1e9dd5","Type":"ContainerStarted","Data":"eb5505907b6806de29689af4eb46e8217228f80729d4ab29e116dbb46460dca3"}
Sep 19 10:38:46.253438 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:46.253392    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:46.440004 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:46.439977    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:47.120620 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:47.120585    2579 generic.go:356] "Generic (PLEG): container finished" podID="902a37e7-4fae-4f82-ab3b-df723e1e9dd5" containerID="bc566e29e7850cc1425997cfb07989ddb42378aa236f489d8dbbf627d54a4e51" exitCode=0
Sep 19 10:38:47.120731 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:47.120670    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg" event={"ID":"902a37e7-4fae-4f82-ab3b-df723e1e9dd5","Type":"ContainerDied","Data":"bc566e29e7850cc1425997cfb07989ddb42378aa236f489d8dbbf627d54a4e51"}
Sep 19 10:38:48.126967 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:48.126932    2579 generic.go:356] "Generic (PLEG): container finished" podID="902a37e7-4fae-4f82-ab3b-df723e1e9dd5" containerID="3e0270cdb31b42c49a7cef270106cce77349ea82b7a560eedb0d1c161b3e0e82" exitCode=0
Sep 19 10:38:48.127311 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:48.126982    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg" event={"ID":"902a37e7-4fae-4f82-ab3b-df723e1e9dd5","Type":"ContainerDied","Data":"3e0270cdb31b42c49a7cef270106cce77349ea82b7a560eedb0d1c161b3e0e82"}
Sep 19 10:38:48.925553 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:48.925507    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:48.990555 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:48.990517    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:49.247729 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:49.247703    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"
Sep 19 10:38:49.425572 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:49.425545    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-bundle\") pod \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\" (UID: \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\") "
Sep 19 10:38:49.425728 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:49.425594    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-util\") pod \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\" (UID: \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\") "
Sep 19 10:38:49.425728 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:49.425643    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbnx6\" (UniqueName: \"kubernetes.io/projected/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-kube-api-access-xbnx6\") pod \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\" (UID: \"902a37e7-4fae-4f82-ab3b-df723e1e9dd5\") "
Sep 19 10:38:49.425942 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:49.425922    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-bundle" (OuterVolumeSpecName: "bundle") pod "902a37e7-4fae-4f82-ab3b-df723e1e9dd5" (UID: "902a37e7-4fae-4f82-ab3b-df723e1e9dd5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:38:49.427861 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:49.427791    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-kube-api-access-xbnx6" (OuterVolumeSpecName: "kube-api-access-xbnx6") pod "902a37e7-4fae-4f82-ab3b-df723e1e9dd5" (UID: "902a37e7-4fae-4f82-ab3b-df723e1e9dd5"). InnerVolumeSpecName "kube-api-access-xbnx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:38:49.431167 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:49.431142    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-util" (OuterVolumeSpecName: "util") pod "902a37e7-4fae-4f82-ab3b-df723e1e9dd5" (UID: "902a37e7-4fae-4f82-ab3b-df723e1e9dd5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:38:49.526624 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:49.526594    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-util\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:38:49.526725 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:49.526643    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbnx6\" (UniqueName: \"kubernetes.io/projected/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-kube-api-access-xbnx6\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:38:49.526725 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:49.526658    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/902a37e7-4fae-4f82-ab3b-df723e1e9dd5-bundle\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:38:50.136617 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:50.136586    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg"
Sep 19 10:38:50.136777 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:50.136587    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fghhvg" event={"ID":"902a37e7-4fae-4f82-ab3b-df723e1e9dd5","Type":"ContainerDied","Data":"eb5505907b6806de29689af4eb46e8217228f80729d4ab29e116dbb46460dca3"}
Sep 19 10:38:50.136777 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:50.136688    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb5505907b6806de29689af4eb46e8217228f80729d4ab29e116dbb46460dca3"
Sep 19 10:38:53.716101 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:53.716017    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:53.806376 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:53.806348    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:54.454665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.454635    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2bph9"]
Sep 19 10:38:54.454913 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.454902    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="902a37e7-4fae-4f82-ab3b-df723e1e9dd5" containerName="extract"
Sep 19 10:38:54.454956 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.454915    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="902a37e7-4fae-4f82-ab3b-df723e1e9dd5" containerName="extract"
Sep 19 10:38:54.454956 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.454926    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="902a37e7-4fae-4f82-ab3b-df723e1e9dd5" containerName="pull"
Sep 19 10:38:54.454956 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.454931    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="902a37e7-4fae-4f82-ab3b-df723e1e9dd5" containerName="pull"
Sep 19 10:38:54.454956 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.454940    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="902a37e7-4fae-4f82-ab3b-df723e1e9dd5" containerName="util"
Sep 19 10:38:54.454956 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.454946    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="902a37e7-4fae-4f82-ab3b-df723e1e9dd5" containerName="util"
Sep 19 10:38:54.455097 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.454985    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="902a37e7-4fae-4f82-ab3b-df723e1e9dd5" containerName="extract"
Sep 19 10:38:54.456534 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.456509    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-2bph9"
Sep 19 10:38:54.457345 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.457325    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:54.459065 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.459048    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-69qpw\""
Sep 19 10:38:54.459264 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.459248    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 19 10:38:54.460209 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.460192    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 19 10:38:54.467651 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.467622    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2bph9"]
Sep 19 10:38:54.559735 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.559712    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmkvp\" (UniqueName: \"kubernetes.io/projected/39158f9d-9fdd-4f02-9b13-7daeebe35ba0-kube-api-access-hmkvp\") pod \"cert-manager-55b4d954d5-2bph9\" (UID: \"39158f9d-9fdd-4f02-9b13-7daeebe35ba0\") " pod="cert-manager/cert-manager-55b4d954d5-2bph9"
Sep 19 10:38:54.559858 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.559770    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39158f9d-9fdd-4f02-9b13-7daeebe35ba0-bound-sa-token\") pod \"cert-manager-55b4d954d5-2bph9\" (UID: \"39158f9d-9fdd-4f02-9b13-7daeebe35ba0\") " pod="cert-manager/cert-manager-55b4d954d5-2bph9"
Sep 19 10:38:54.661055 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.661034    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmkvp\" (UniqueName: \"kubernetes.io/projected/39158f9d-9fdd-4f02-9b13-7daeebe35ba0-kube-api-access-hmkvp\") pod \"cert-manager-55b4d954d5-2bph9\" (UID: \"39158f9d-9fdd-4f02-9b13-7daeebe35ba0\") " pod="cert-manager/cert-manager-55b4d954d5-2bph9"
Sep 19 10:38:54.661172 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.661088    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39158f9d-9fdd-4f02-9b13-7daeebe35ba0-bound-sa-token\") pod \"cert-manager-55b4d954d5-2bph9\" (UID: \"39158f9d-9fdd-4f02-9b13-7daeebe35ba0\") " pod="cert-manager/cert-manager-55b4d954d5-2bph9"
Sep 19 10:38:54.669347 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.669327    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39158f9d-9fdd-4f02-9b13-7daeebe35ba0-bound-sa-token\") pod \"cert-manager-55b4d954d5-2bph9\" (UID: \"39158f9d-9fdd-4f02-9b13-7daeebe35ba0\") " pod="cert-manager/cert-manager-55b4d954d5-2bph9"
Sep 19 10:38:54.669454 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.669434    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmkvp\" (UniqueName: \"kubernetes.io/projected/39158f9d-9fdd-4f02-9b13-7daeebe35ba0-kube-api-access-hmkvp\") pod \"cert-manager-55b4d954d5-2bph9\" (UID: \"39158f9d-9fdd-4f02-9b13-7daeebe35ba0\") " pod="cert-manager/cert-manager-55b4d954d5-2bph9"
Sep 19 10:38:54.765552 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:54.765478    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-2bph9"
Sep 19 10:38:55.030032 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:55.030007    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2bph9"]
Sep 19 10:38:55.032259 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:38:55.032232    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39158f9d_9fdd_4f02_9b13_7daeebe35ba0.slice/crio-0e0330949d97c3f126f06676f282e8590a30f42b18d52687bdafa7edf6df2705 WatchSource:0}: Error finding container 0e0330949d97c3f126f06676f282e8590a30f42b18d52687bdafa7edf6df2705: Status 404 returned error can't find the container with id 0e0330949d97c3f126f06676f282e8590a30f42b18d52687bdafa7edf6df2705
Sep 19 10:38:55.155494 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:55.155466    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-2bph9" event={"ID":"39158f9d-9fdd-4f02-9b13-7daeebe35ba0","Type":"ContainerStarted","Data":"0e0330949d97c3f126f06676f282e8590a30f42b18d52687bdafa7edf6df2705"}
Sep 19 10:38:55.333971 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:55.333895    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:55.433797 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:55.433765    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:38:58.170633 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:58.170597    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-2bph9" event={"ID":"39158f9d-9fdd-4f02-9b13-7daeebe35ba0","Type":"ContainerStarted","Data":"ecd6c0ab0d5e9300da570fa15a4ebec3947e1f3e862f6b89df094d6474951937"}
Sep 19 10:38:58.187101 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:38:58.187045    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-2bph9" podStartSLOduration=1.765036211 podStartE2EDuration="4.187027576s" totalImagesPullingTime="2.421991365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:38:58.185288693 +0000 UTC m=+259.096982212" watchObservedRunningTime="2026-09-19 10:38:58.187027576 +0000 UTC m=+259.098721097"
Sep 19 10:39:01.961462 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:01.961387    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:02.062373 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:02.062338    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:03.661853 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:03.661794    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:04.740486 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.740452    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"]
Sep 19 10:39:04.742751 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.742736    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"
Sep 19 10:39:04.745455 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.745435    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 10:39:04.745567 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.745461    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 10:39:04.746702 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.746688    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rk9vp\""
Sep 19 10:39:04.752761 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.752740    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"]
Sep 19 10:39:04.836862 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.836842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6311ffe3-dbd3-43e7-bce0-6972978f4be0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96\" (UID: \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"
Sep 19 10:39:04.836975 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.836874    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s228\" (UniqueName: \"kubernetes.io/projected/6311ffe3-dbd3-43e7-bce0-6972978f4be0-kube-api-access-7s228\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96\" (UID: \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"
Sep 19 10:39:04.836975 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.836898    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6311ffe3-dbd3-43e7-bce0-6972978f4be0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96\" (UID: \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"
Sep 19 10:39:04.937516 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.937491    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6311ffe3-dbd3-43e7-bce0-6972978f4be0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96\" (UID: \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"
Sep 19 10:39:04.937684 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.937539    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s228\" (UniqueName: \"kubernetes.io/projected/6311ffe3-dbd3-43e7-bce0-6972978f4be0-kube-api-access-7s228\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96\" (UID: \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"
Sep 19 10:39:04.937684 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.937576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6311ffe3-dbd3-43e7-bce0-6972978f4be0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96\" (UID: \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"
Sep 19 10:39:04.937915 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.937894    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6311ffe3-dbd3-43e7-bce0-6972978f4be0-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96\" (UID: \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"
Sep 19 10:39:04.937954 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.937910    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6311ffe3-dbd3-43e7-bce0-6972978f4be0-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96\" (UID: \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"
Sep 19 10:39:04.946215 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:04.946191    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s228\" (UniqueName: \"kubernetes.io/projected/6311ffe3-dbd3-43e7-bce0-6972978f4be0-kube-api-access-7s228\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96\" (UID: \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"
Sep 19 10:39:05.052039 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:05.051989    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"
Sep 19 10:39:05.350443 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:39:05.350412    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6311ffe3_dbd3_43e7_bce0_6972978f4be0.slice/crio-a639d5997f64207b3cffccf68c24082fba262a1ddc87c7462ca8984c49cee2b9 WatchSource:0}: Error finding container a639d5997f64207b3cffccf68c24082fba262a1ddc87c7462ca8984c49cee2b9: Status 404 returned error can't find the container with id a639d5997f64207b3cffccf68c24082fba262a1ddc87c7462ca8984c49cee2b9
Sep 19 10:39:05.356175 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:05.356153    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"]
Sep 19 10:39:06.202056 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:06.201974    2579 generic.go:356] "Generic (PLEG): container finished" podID="6311ffe3-dbd3-43e7-bce0-6972978f4be0" containerID="af7a280b46d7ea3cbdf565d5946eadcc6bf2237149ee86dc5e2f3c6a85a726e8" exitCode=0
Sep 19 10:39:06.202056 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:06.202022    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96" event={"ID":"6311ffe3-dbd3-43e7-bce0-6972978f4be0","Type":"ContainerDied","Data":"af7a280b46d7ea3cbdf565d5946eadcc6bf2237149ee86dc5e2f3c6a85a726e8"}
Sep 19 10:39:06.202416 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:06.202075    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96" event={"ID":"6311ffe3-dbd3-43e7-bce0-6972978f4be0","Type":"ContainerStarted","Data":"a639d5997f64207b3cffccf68c24082fba262a1ddc87c7462ca8984c49cee2b9"}
Sep 19 10:39:07.207470 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:07.207443    2579 generic.go:356] "Generic (PLEG): container finished" podID="6311ffe3-dbd3-43e7-bce0-6972978f4be0" containerID="c34cd814b4741f158fb94bd5708ce9dff532c613d2b6f2bd4724684bcbd67c5d" exitCode=0
Sep 19 10:39:07.207960 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:07.207497    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96" event={"ID":"6311ffe3-dbd3-43e7-bce0-6972978f4be0","Type":"ContainerDied","Data":"c34cd814b4741f158fb94bd5708ce9dff532c613d2b6f2bd4724684bcbd67c5d"}
Sep 19 10:39:08.213888 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:08.213855    2579 generic.go:356] "Generic (PLEG): container finished" podID="6311ffe3-dbd3-43e7-bce0-6972978f4be0" containerID="b0686ddfd63dfdde26fb6f60199ca65f63484527e7372743f48fc7e36a90acf7" exitCode=0
Sep 19 10:39:08.214291 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:08.213949    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96" event={"ID":"6311ffe3-dbd3-43e7-bce0-6972978f4be0","Type":"ContainerDied","Data":"b0686ddfd63dfdde26fb6f60199ca65f63484527e7372743f48fc7e36a90acf7"}
Sep 19 10:39:09.340253 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:09.340229    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"
Sep 19 10:39:09.378883 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:09.378859    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6311ffe3-dbd3-43e7-bce0-6972978f4be0-bundle\") pod \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\" (UID: \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\") "
Sep 19 10:39:09.379020 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:09.378921    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6311ffe3-dbd3-43e7-bce0-6972978f4be0-util\") pod \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\" (UID: \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\") "
Sep 19 10:39:09.379020 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:09.378949    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7s228\" (UniqueName: \"kubernetes.io/projected/6311ffe3-dbd3-43e7-bce0-6972978f4be0-kube-api-access-7s228\") pod \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\" (UID: \"6311ffe3-dbd3-43e7-bce0-6972978f4be0\") "
Sep 19 10:39:09.379587 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:09.379556    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6311ffe3-dbd3-43e7-bce0-6972978f4be0-bundle" (OuterVolumeSpecName: "bundle") pod "6311ffe3-dbd3-43e7-bce0-6972978f4be0" (UID: "6311ffe3-dbd3-43e7-bce0-6972978f4be0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:39:09.380983 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:09.380951    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6311ffe3-dbd3-43e7-bce0-6972978f4be0-kube-api-access-7s228" (OuterVolumeSpecName: "kube-api-access-7s228") pod "6311ffe3-dbd3-43e7-bce0-6972978f4be0" (UID: "6311ffe3-dbd3-43e7-bce0-6972978f4be0"). InnerVolumeSpecName "kube-api-access-7s228". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:39:09.384708 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:09.384691    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6311ffe3-dbd3-43e7-bce0-6972978f4be0-util" (OuterVolumeSpecName: "util") pod "6311ffe3-dbd3-43e7-bce0-6972978f4be0" (UID: "6311ffe3-dbd3-43e7-bce0-6972978f4be0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:39:09.479736 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:09.479672    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6311ffe3-dbd3-43e7-bce0-6972978f4be0-util\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:39:09.479736 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:09.479701    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7s228\" (UniqueName: \"kubernetes.io/projected/6311ffe3-dbd3-43e7-bce0-6972978f4be0-kube-api-access-7s228\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:39:09.479736 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:09.479711    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6311ffe3-dbd3-43e7-bce0-6972978f4be0-bundle\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:39:10.223847 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:10.223815    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96"
Sep 19 10:39:10.223847 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:10.223836    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4jf96" event={"ID":"6311ffe3-dbd3-43e7-bce0-6972978f4be0","Type":"ContainerDied","Data":"a639d5997f64207b3cffccf68c24082fba262a1ddc87c7462ca8984c49cee2b9"}
Sep 19 10:39:10.224048 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:10.223877    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a639d5997f64207b3cffccf68c24082fba262a1ddc87c7462ca8984c49cee2b9"
Sep 19 10:39:14.766155 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.766118    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"]
Sep 19 10:39:14.766508 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.766435    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6311ffe3-dbd3-43e7-bce0-6972978f4be0" containerName="extract"
Sep 19 10:39:14.766508 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.766447    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6311ffe3-dbd3-43e7-bce0-6972978f4be0" containerName="extract"
Sep 19 10:39:14.766508 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.766458    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6311ffe3-dbd3-43e7-bce0-6972978f4be0" containerName="pull"
Sep 19 10:39:14.766508 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.766464    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6311ffe3-dbd3-43e7-bce0-6972978f4be0" containerName="pull"
Sep 19 10:39:14.766508 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.766472    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6311ffe3-dbd3-43e7-bce0-6972978f4be0" containerName="util"
Sep 19 10:39:14.766508 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.766478    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6311ffe3-dbd3-43e7-bce0-6972978f4be0" containerName="util"
Sep 19 10:39:14.766757 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.766544    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6311ffe3-dbd3-43e7-bce0-6972978f4be0" containerName="extract"
Sep 19 10:39:14.772258 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.772237    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"
Sep 19 10:39:14.774929 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.774909    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rk9vp\""
Sep 19 10:39:14.775040 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.774909    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 10:39:14.776153 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.776131    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 10:39:14.778442 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.778423    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"]
Sep 19 10:39:14.816466 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.816438    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20b34550-c491-4994-8ea3-56eee61a9bc9-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8\" (UID: \"20b34550-c491-4994-8ea3-56eee61a9bc9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"
Sep 19 10:39:14.816599 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.816479    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20b34550-c491-4994-8ea3-56eee61a9bc9-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8\" (UID: \"20b34550-c491-4994-8ea3-56eee61a9bc9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"
Sep 19 10:39:14.816599 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.816506    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k2l9\" (UniqueName: \"kubernetes.io/projected/20b34550-c491-4994-8ea3-56eee61a9bc9-kube-api-access-9k2l9\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8\" (UID: \"20b34550-c491-4994-8ea3-56eee61a9bc9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"
Sep 19 10:39:14.917753 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.917727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20b34550-c491-4994-8ea3-56eee61a9bc9-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8\" (UID: \"20b34550-c491-4994-8ea3-56eee61a9bc9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"
Sep 19 10:39:14.917753 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.917757    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k2l9\" (UniqueName: \"kubernetes.io/projected/20b34550-c491-4994-8ea3-56eee61a9bc9-kube-api-access-9k2l9\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8\" (UID: \"20b34550-c491-4994-8ea3-56eee61a9bc9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"
Sep 19 10:39:14.917938 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.917811    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20b34550-c491-4994-8ea3-56eee61a9bc9-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8\" (UID: \"20b34550-c491-4994-8ea3-56eee61a9bc9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"
Sep 19 10:39:14.918115 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.918097    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20b34550-c491-4994-8ea3-56eee61a9bc9-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8\" (UID: \"20b34550-c491-4994-8ea3-56eee61a9bc9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"
Sep 19 10:39:14.918152 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.918137    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20b34550-c491-4994-8ea3-56eee61a9bc9-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8\" (UID: \"20b34550-c491-4994-8ea3-56eee61a9bc9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"
Sep 19 10:39:14.926372 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:14.926350    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k2l9\" (UniqueName: \"kubernetes.io/projected/20b34550-c491-4994-8ea3-56eee61a9bc9-kube-api-access-9k2l9\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8\" (UID: \"20b34550-c491-4994-8ea3-56eee61a9bc9\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"
Sep 19 10:39:15.082822 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:15.082741    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"
Sep 19 10:39:15.353327 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:15.353303    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"]
Sep 19 10:39:15.355703 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:39:15.355673    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20b34550_c491_4994_8ea3_56eee61a9bc9.slice/crio-5854e82b1b48745fcdc896aa40cafa66bf1265a75170c6721147ef029ea47d4f WatchSource:0}: Error finding container 5854e82b1b48745fcdc896aa40cafa66bf1265a75170c6721147ef029ea47d4f: Status 404 returned error can't find the container with id 5854e82b1b48745fcdc896aa40cafa66bf1265a75170c6721147ef029ea47d4f
Sep 19 10:39:15.546919 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:15.546889    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:15.667055 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:15.666967    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:15.961376 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:15.961343    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:16.217638 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.217560    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:16.249762 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.249733    2579 generic.go:356] "Generic (PLEG): container finished" podID="20b34550-c491-4994-8ea3-56eee61a9bc9" containerID="0b49e1fd29420227760cea69245e627c8fd5dc3aa5131da732d5df67dffa0957" exitCode=0
Sep 19 10:39:16.249884 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.249819    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8" event={"ID":"20b34550-c491-4994-8ea3-56eee61a9bc9","Type":"ContainerDied","Data":"0b49e1fd29420227760cea69245e627c8fd5dc3aa5131da732d5df67dffa0957"}
Sep 19 10:39:16.249884 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.249866    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8" event={"ID":"20b34550-c491-4994-8ea3-56eee61a9bc9","Type":"ContainerStarted","Data":"5854e82b1b48745fcdc896aa40cafa66bf1265a75170c6721147ef029ea47d4f"}
Sep 19 10:39:16.322205 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.322174    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:16.332830 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.332804    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"]
Sep 19 10:39:16.336539 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.336510    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.339096 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.339076    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 19 10:39:16.339312 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.339294    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 19 10:39:16.339465 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.339442    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bmfwr\""
Sep 19 10:39:16.339794 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.339547    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 19 10:39:16.339794 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.339790    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 19 10:39:16.357154 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.357133    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"]
Sep 19 10:39:16.428818 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.428790    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/51b7401b-5d3d-4fdf-a9ea-8d735e333095-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.428921 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.428834    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51b7401b-5d3d-4fdf-a9ea-8d735e333095-tmp\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.428921 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.428858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/51b7401b-5d3d-4fdf-a9ea-8d735e333095-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.429022 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.428920    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/51b7401b-5d3d-4fdf-a9ea-8d735e333095-manifests\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.429022 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.428988    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5xnm\" (UniqueName: \"kubernetes.io/projected/51b7401b-5d3d-4fdf-a9ea-8d735e333095-kube-api-access-f5xnm\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.527004 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.526943    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:16.530030 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.530010    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5xnm\" (UniqueName: \"kubernetes.io/projected/51b7401b-5d3d-4fdf-a9ea-8d735e333095-kube-api-access-f5xnm\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.530112 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.530043    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/51b7401b-5d3d-4fdf-a9ea-8d735e333095-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.530112 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.530069    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51b7401b-5d3d-4fdf-a9ea-8d735e333095-tmp\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.530112 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.530088    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/51b7401b-5d3d-4fdf-a9ea-8d735e333095-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.530246 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.530134    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/51b7401b-5d3d-4fdf-a9ea-8d735e333095-manifests\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.530538 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.530510    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/51b7401b-5d3d-4fdf-a9ea-8d735e333095-manifests\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.532424 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.532394    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51b7401b-5d3d-4fdf-a9ea-8d735e333095-tmp\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.532579 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.532563    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/51b7401b-5d3d-4fdf-a9ea-8d735e333095-webhook-cert\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.532663 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.532645    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/51b7401b-5d3d-4fdf-a9ea-8d735e333095-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.541613 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.541590    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5xnm\" (UniqueName: \"kubernetes.io/projected/51b7401b-5d3d-4fdf-a9ea-8d735e333095-kube-api-access-f5xnm\") pod \"opendatahub-operator-controller-manager-9b87889df-lwjlp\" (UID: \"51b7401b-5d3d-4fdf-a9ea-8d735e333095\") " pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.633033 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.633010    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:16.646630 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.646609    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:16.738667 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.738642    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:16.826944 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.826880    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:16.921631 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.921616    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"]
Sep 19 10:39:16.923791 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:39:16.923763    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51b7401b_5d3d_4fdf_a9ea_8d735e333095.slice/crio-d7ee4f67e884ba0ed34506757b4665285f7a39c6df1a957a8dd39c50ff8aba3b WatchSource:0}: Error finding container d7ee4f67e884ba0ed34506757b4665285f7a39c6df1a957a8dd39c50ff8aba3b: Status 404 returned error can't find the container with id d7ee4f67e884ba0ed34506757b4665285f7a39c6df1a957a8dd39c50ff8aba3b
Sep 19 10:39:16.936337 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:16.936313    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:17.015720 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:17.015699    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:17.258372 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:17.258340    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp" event={"ID":"51b7401b-5d3d-4fdf-a9ea-8d735e333095","Type":"ContainerStarted","Data":"d7ee4f67e884ba0ed34506757b4665285f7a39c6df1a957a8dd39c50ff8aba3b"}
Sep 19 10:39:18.266383 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:18.266302    2579 generic.go:356] "Generic (PLEG): container finished" podID="20b34550-c491-4994-8ea3-56eee61a9bc9" containerID="d3cbb32a62a59627e1085830813b8d115b5e8f64de302ad95ded0a72fda46f00" exitCode=0
Sep 19 10:39:18.266815 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:18.266383    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8" event={"ID":"20b34550-c491-4994-8ea3-56eee61a9bc9","Type":"ContainerDied","Data":"d3cbb32a62a59627e1085830813b8d115b5e8f64de302ad95ded0a72fda46f00"}
Sep 19 10:39:20.280084 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:20.280049    2579 generic.go:356] "Generic (PLEG): container finished" podID="20b34550-c491-4994-8ea3-56eee61a9bc9" containerID="eb88f794723254a63b0ad34f1fcfd8d2a10db8690237df3d05e14e9532c4a89c" exitCode=0
Sep 19 10:39:20.280509 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:20.280126    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8" event={"ID":"20b34550-c491-4994-8ea3-56eee61a9bc9","Type":"ContainerDied","Data":"eb88f794723254a63b0ad34f1fcfd8d2a10db8690237df3d05e14e9532c4a89c"}
Sep 19 10:39:20.281644 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:20.281621    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp" event={"ID":"51b7401b-5d3d-4fdf-a9ea-8d735e333095","Type":"ContainerStarted","Data":"3d30205ebd1d715c6043a8ca068b48d6ecbf7b9d4c50e613a6e05acf76f25476"}
Sep 19 10:39:21.286963 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:21.286880    2579 generic.go:356] "Generic (PLEG): container finished" podID="51b7401b-5d3d-4fdf-a9ea-8d735e333095" containerID="3d30205ebd1d715c6043a8ca068b48d6ecbf7b9d4c50e613a6e05acf76f25476" exitCode=0
Sep 19 10:39:21.287318 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:21.286959    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp" event={"ID":"51b7401b-5d3d-4fdf-a9ea-8d735e333095","Type":"ContainerDied","Data":"3d30205ebd1d715c6043a8ca068b48d6ecbf7b9d4c50e613a6e05acf76f25476"}
Sep 19 10:39:21.415248 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:21.415220    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"
Sep 19 10:39:21.473940 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:21.473904    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20b34550-c491-4994-8ea3-56eee61a9bc9-bundle\") pod \"20b34550-c491-4994-8ea3-56eee61a9bc9\" (UID: \"20b34550-c491-4994-8ea3-56eee61a9bc9\") "
Sep 19 10:39:21.474145 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:21.473950    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9k2l9\" (UniqueName: \"kubernetes.io/projected/20b34550-c491-4994-8ea3-56eee61a9bc9-kube-api-access-9k2l9\") pod \"20b34550-c491-4994-8ea3-56eee61a9bc9\" (UID: \"20b34550-c491-4994-8ea3-56eee61a9bc9\") "
Sep 19 10:39:21.474145 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:21.473974    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20b34550-c491-4994-8ea3-56eee61a9bc9-util\") pod \"20b34550-c491-4994-8ea3-56eee61a9bc9\" (UID: \"20b34550-c491-4994-8ea3-56eee61a9bc9\") "
Sep 19 10:39:21.474658 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:21.474628    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20b34550-c491-4994-8ea3-56eee61a9bc9-bundle" (OuterVolumeSpecName: "bundle") pod "20b34550-c491-4994-8ea3-56eee61a9bc9" (UID: "20b34550-c491-4994-8ea3-56eee61a9bc9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:39:21.476064 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:21.476033    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20b34550-c491-4994-8ea3-56eee61a9bc9-kube-api-access-9k2l9" (OuterVolumeSpecName: "kube-api-access-9k2l9") pod "20b34550-c491-4994-8ea3-56eee61a9bc9" (UID: "20b34550-c491-4994-8ea3-56eee61a9bc9"). InnerVolumeSpecName "kube-api-access-9k2l9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:39:21.478241 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:21.478204    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20b34550-c491-4994-8ea3-56eee61a9bc9-util" (OuterVolumeSpecName: "util") pod "20b34550-c491-4994-8ea3-56eee61a9bc9" (UID: "20b34550-c491-4994-8ea3-56eee61a9bc9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:39:21.575378 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:21.575291    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/20b34550-c491-4994-8ea3-56eee61a9bc9-bundle\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:39:21.575378 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:21.575325    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9k2l9\" (UniqueName: \"kubernetes.io/projected/20b34550-c491-4994-8ea3-56eee61a9bc9-kube-api-access-9k2l9\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:39:21.575378 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:21.575335    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/20b34550-c491-4994-8ea3-56eee61a9bc9-util\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:39:22.294223 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:22.294188    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8"
Sep 19 10:39:22.294647 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:22.294220    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0tcbz8" event={"ID":"20b34550-c491-4994-8ea3-56eee61a9bc9","Type":"ContainerDied","Data":"5854e82b1b48745fcdc896aa40cafa66bf1265a75170c6721147ef029ea47d4f"}
Sep 19 10:39:22.294647 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:22.294272    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5854e82b1b48745fcdc896aa40cafa66bf1265a75170c6721147ef029ea47d4f"
Sep 19 10:39:22.296567 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:22.296536    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp" event={"ID":"51b7401b-5d3d-4fdf-a9ea-8d735e333095","Type":"ContainerStarted","Data":"9b2b6c3f45c6dc8f0203dfc72cda2bd3c23bd6ace69d1ef5baf823da9f280ae5"}
Sep 19 10:39:22.296709 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:22.296695    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:22.323794 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:22.323737    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp" podStartSLOduration=1.9845925260000001 podStartE2EDuration="6.323718437s" totalImagesPullingTime="3.303548139s" totalInitContainerRuntime="1.035577772s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:39:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:39:22.321708875 +0000 UTC m=+283.233402396" watchObservedRunningTime="2026-09-19 10:39:22.323718437 +0000 UTC m=+283.235411955"
Sep 19 10:39:29.902436 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:29.902390    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:30.001884 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:30.001850    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:33.303971 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:33.303942    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9b87889df-lwjlp"
Sep 19 10:39:35.130849 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.130819    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:35.533951 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.533919    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk"]
Sep 19 10:39:35.534350 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.534325    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20b34550-c491-4994-8ea3-56eee61a9bc9" containerName="extract"
Sep 19 10:39:35.534350 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.534348    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="20b34550-c491-4994-8ea3-56eee61a9bc9" containerName="extract"
Sep 19 10:39:35.534567 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.534389    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20b34550-c491-4994-8ea3-56eee61a9bc9" containerName="util"
Sep 19 10:39:35.534567 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.534399    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="20b34550-c491-4994-8ea3-56eee61a9bc9" containerName="util"
Sep 19 10:39:35.534567 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.534408    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20b34550-c491-4994-8ea3-56eee61a9bc9" containerName="pull"
Sep 19 10:39:35.534567 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.534416    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="20b34550-c491-4994-8ea3-56eee61a9bc9" containerName="pull"
Sep 19 10:39:35.534567 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.534506    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="20b34550-c491-4994-8ea3-56eee61a9bc9" containerName="extract"
Sep 19 10:39:35.540014 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.539996    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk"
Sep 19 10:39:35.543160 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.543140    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 19 10:39:35.543278 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.543143    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 19 10:39:35.543422 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.543402    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 19 10:39:35.544802 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.544788    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 19 10:39:35.561073 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.561045    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk"]
Sep 19 10:39:35.675274 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.675242    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbpmm\" (UniqueName: \"kubernetes.io/projected/4af6e02a-6920-412e-9baa-86ff466cdf24-kube-api-access-nbpmm\") pod \"kube-auth-proxy-8b786b9b5-wppgk\" (UID: \"4af6e02a-6920-412e-9baa-86ff466cdf24\") " pod="openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk"
Sep 19 10:39:35.675503 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.675298    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4af6e02a-6920-412e-9baa-86ff466cdf24-tmp\") pod \"kube-auth-proxy-8b786b9b5-wppgk\" (UID: \"4af6e02a-6920-412e-9baa-86ff466cdf24\") " pod="openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk"
Sep 19 10:39:35.675503 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.675321    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4af6e02a-6920-412e-9baa-86ff466cdf24-tls-certs\") pod \"kube-auth-proxy-8b786b9b5-wppgk\" (UID: \"4af6e02a-6920-412e-9baa-86ff466cdf24\") " pod="openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk"
Sep 19 10:39:35.775850 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.775822    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4af6e02a-6920-412e-9baa-86ff466cdf24-tls-certs\") pod \"kube-auth-proxy-8b786b9b5-wppgk\" (UID: \"4af6e02a-6920-412e-9baa-86ff466cdf24\") " pod="openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk"
Sep 19 10:39:35.775994 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.775867    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbpmm\" (UniqueName: \"kubernetes.io/projected/4af6e02a-6920-412e-9baa-86ff466cdf24-kube-api-access-nbpmm\") pod \"kube-auth-proxy-8b786b9b5-wppgk\" (UID: \"4af6e02a-6920-412e-9baa-86ff466cdf24\") " pod="openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk"
Sep 19 10:39:35.775994 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.775907    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4af6e02a-6920-412e-9baa-86ff466cdf24-tmp\") pod \"kube-auth-proxy-8b786b9b5-wppgk\" (UID: \"4af6e02a-6920-412e-9baa-86ff466cdf24\") " pod="openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk"
Sep 19 10:39:35.777988 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.777966    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4af6e02a-6920-412e-9baa-86ff466cdf24-tmp\") pod \"kube-auth-proxy-8b786b9b5-wppgk\" (UID: \"4af6e02a-6920-412e-9baa-86ff466cdf24\") " pod="openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk"
Sep 19 10:39:35.778315 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.778296    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4af6e02a-6920-412e-9baa-86ff466cdf24-tls-certs\") pod \"kube-auth-proxy-8b786b9b5-wppgk\" (UID: \"4af6e02a-6920-412e-9baa-86ff466cdf24\") " pod="openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk"
Sep 19 10:39:35.789240 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.789191    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbpmm\" (UniqueName: \"kubernetes.io/projected/4af6e02a-6920-412e-9baa-86ff466cdf24-kube-api-access-nbpmm\") pod \"kube-auth-proxy-8b786b9b5-wppgk\" (UID: \"4af6e02a-6920-412e-9baa-86ff466cdf24\") " pod="openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk"
Sep 19 10:39:35.848705 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:35.848682    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk"
Sep 19 10:39:36.123905 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.123881    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk"]
Sep 19 10:39:36.125928 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:39:36.125894    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4af6e02a_6920_412e_9baa_86ff466cdf24.slice/crio-7486f5e1fd6d99fa8c1df8a7e0f0a6e777f6ff26d547f050e5bf10f048cd61e5 WatchSource:0}: Error finding container 7486f5e1fd6d99fa8c1df8a7e0f0a6e777f6ff26d547f050e5bf10f048cd61e5: Status 404 returned error can't find the container with id 7486f5e1fd6d99fa8c1df8a7e0f0a6e777f6ff26d547f050e5bf10f048cd61e5
Sep 19 10:39:36.195259 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.195226    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"]
Sep 19 10:39:36.199770 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.199751    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:36.203202 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.203179    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 19 10:39:36.203290 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.203196    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 19 10:39:36.203290 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.203237    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 19 10:39:36.204387 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.204372    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 19 10:39:36.204387 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.204378    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mm8vq\""
Sep 19 10:39:36.204503 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.204407    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 19 10:39:36.216900 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.216879    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"]
Sep 19 10:39:36.280768 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.280745    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3fd3981-41ea-4832-a60f-b1ad893fd18f-cert\") pod \"lws-controller-manager-6c7d4469d9-2j6qn\" (UID: \"d3fd3981-41ea-4832-a60f-b1ad893fd18f\") " pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:36.280897 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.280778    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3fd3981-41ea-4832-a60f-b1ad893fd18f-metrics-cert\") pod \"lws-controller-manager-6c7d4469d9-2j6qn\" (UID: \"d3fd3981-41ea-4832-a60f-b1ad893fd18f\") " pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:36.280897 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.280813    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x7cj\" (UniqueName: \"kubernetes.io/projected/d3fd3981-41ea-4832-a60f-b1ad893fd18f-kube-api-access-4x7cj\") pod \"lws-controller-manager-6c7d4469d9-2j6qn\" (UID: \"d3fd3981-41ea-4832-a60f-b1ad893fd18f\") " pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:36.281022 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.280900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d3fd3981-41ea-4832-a60f-b1ad893fd18f-manager-config\") pod \"lws-controller-manager-6c7d4469d9-2j6qn\" (UID: \"d3fd3981-41ea-4832-a60f-b1ad893fd18f\") " pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:36.364644 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.364619    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk" event={"ID":"4af6e02a-6920-412e-9baa-86ff466cdf24","Type":"ContainerStarted","Data":"7486f5e1fd6d99fa8c1df8a7e0f0a6e777f6ff26d547f050e5bf10f048cd61e5"}
Sep 19 10:39:36.381996 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.381939    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3fd3981-41ea-4832-a60f-b1ad893fd18f-cert\") pod \"lws-controller-manager-6c7d4469d9-2j6qn\" (UID: \"d3fd3981-41ea-4832-a60f-b1ad893fd18f\") " pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:36.381996 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.381964    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3fd3981-41ea-4832-a60f-b1ad893fd18f-metrics-cert\") pod \"lws-controller-manager-6c7d4469d9-2j6qn\" (UID: \"d3fd3981-41ea-4832-a60f-b1ad893fd18f\") " pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:36.381996 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.381987    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x7cj\" (UniqueName: \"kubernetes.io/projected/d3fd3981-41ea-4832-a60f-b1ad893fd18f-kube-api-access-4x7cj\") pod \"lws-controller-manager-6c7d4469d9-2j6qn\" (UID: \"d3fd3981-41ea-4832-a60f-b1ad893fd18f\") " pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:36.382205 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.382015    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d3fd3981-41ea-4832-a60f-b1ad893fd18f-manager-config\") pod \"lws-controller-manager-6c7d4469d9-2j6qn\" (UID: \"d3fd3981-41ea-4832-a60f-b1ad893fd18f\") " pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:36.382668 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.382649    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d3fd3981-41ea-4832-a60f-b1ad893fd18f-manager-config\") pod \"lws-controller-manager-6c7d4469d9-2j6qn\" (UID: \"d3fd3981-41ea-4832-a60f-b1ad893fd18f\") " pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:36.384338 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.384319    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3fd3981-41ea-4832-a60f-b1ad893fd18f-cert\") pod \"lws-controller-manager-6c7d4469d9-2j6qn\" (UID: \"d3fd3981-41ea-4832-a60f-b1ad893fd18f\") " pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:36.384469 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.384454    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3fd3981-41ea-4832-a60f-b1ad893fd18f-metrics-cert\") pod \"lws-controller-manager-6c7d4469d9-2j6qn\" (UID: \"d3fd3981-41ea-4832-a60f-b1ad893fd18f\") " pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:36.391701 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.391683    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x7cj\" (UniqueName: \"kubernetes.io/projected/d3fd3981-41ea-4832-a60f-b1ad893fd18f-kube-api-access-4x7cj\") pod \"lws-controller-manager-6c7d4469d9-2j6qn\" (UID: \"d3fd3981-41ea-4832-a60f-b1ad893fd18f\") " pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:36.423315 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.423290    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:36.509117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.509095    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:36.789664 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:36.789636    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"]
Sep 19 10:39:36.791153 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:39:36.791109    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3fd3981_41ea_4832_a60f_b1ad893fd18f.slice/crio-35547b08dcb40d9b16072a461c1277a2caad1f3c5ebf8185b0b2e0cd15bdaf30 WatchSource:0}: Error finding container 35547b08dcb40d9b16072a461c1277a2caad1f3c5ebf8185b0b2e0cd15bdaf30: Status 404 returned error can't find the container with id 35547b08dcb40d9b16072a461c1277a2caad1f3c5ebf8185b0b2e0cd15bdaf30
Sep 19 10:39:37.032740 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:37.032702    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:37.105228 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:37.105110    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:37.373472 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:37.373305    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn" event={"ID":"d3fd3981-41ea-4832-a60f-b1ad893fd18f","Type":"ContainerStarted","Data":"35547b08dcb40d9b16072a461c1277a2caad1f3c5ebf8185b0b2e0cd15bdaf30"}
Sep 19 10:39:38.085121 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:38.085083    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:39.385573 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:39.385512    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk" event={"ID":"4af6e02a-6920-412e-9baa-86ff466cdf24","Type":"ContainerStarted","Data":"3bcacef898c6b943f4ad2e12eb319080781b39ddcedae6e899e0bf3baff2711f"}
Sep 19 10:39:39.387160 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:39.387134    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn" event={"ID":"d3fd3981-41ea-4832-a60f-b1ad893fd18f","Type":"ContainerStarted","Data":"d5ee00343d1c8687c57ffb21600822f3d92d2a8d7499853a1c382f8f977f7c03"}
Sep 19 10:39:39.387312 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:39.387266    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:39.401841 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:39.401792    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8b786b9b5-wppgk" podStartSLOduration=1.263161733 podStartE2EDuration="4.401776162s" totalImagesPullingTime="3.138614429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:39:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:39:39.401760045 +0000 UTC m=+300.313453565" watchObservedRunningTime="2026-09-19 10:39:39.401776162 +0000 UTC m=+300.313469681"
Sep 19 10:39:39.419713 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:39.419673    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn" podStartSLOduration=0.951137236 podStartE2EDuration="3.419660586s" totalImagesPullingTime="2.46852335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:39:39.41838447 +0000 UTC m=+300.330077990" watchObservedRunningTime="2026-09-19 10:39:39.419660586 +0000 UTC m=+300.331354099"
Sep 19 10:39:41.028058 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:41.028032    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:39:41.029871 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:41.029850    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:39:41.031584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:41.031567    2579 kubelet.go:1648] "Image garbage collection succeeded"
Sep 19 10:39:42.060104 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:42.060058    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:42.161189 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:42.161158    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:42.341185 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:42.341104    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:42.442238 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:42.442208    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:42.624805 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:42.624735    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:46.282389 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:46.282342    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:46.359791 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:46.359750    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:46.460751 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:46.460722    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:46.561371 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:46.561262    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:46.843863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:46.843775    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:47.008189 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:47.008128    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:50.394745 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:50.394714    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6c7d4469d9-2j6qn"
Sep 19 10:39:55.643861 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:55.643825    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:55.754198 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:55.754172    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:55.840154 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:55.840129    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:55.977905 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:55.977878    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:56.946352 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:56.946315    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:57.101087 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:57.101053    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:39:57.204994 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:39:57.204920    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:40:47.233804 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:47.233666    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:40:47.325706 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:47.325645    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:40:47.497151 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:47.497031    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:40:47.586939 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:47.586911    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:40:49.361817 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:49.361782    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:40:49.461801 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:49.461770    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:40:51.559720 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.559687    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"]
Sep 19 10:40:51.562050 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.562028    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"
Sep 19 10:40:51.565280 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.565258    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 10:40:51.565381 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.565281    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 10:40:51.566732 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.566713    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-rk9vp\""
Sep 19 10:40:51.573395 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.573374    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"]
Sep 19 10:40:51.645691 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.645666    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"]
Sep 19 10:40:51.647932 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.647916    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"
Sep 19 10:40:51.658350 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.658329    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"]
Sep 19 10:40:51.669667 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.669643    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/906df1aa-114e-4d1b-ac8d-3575e3c433d1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k\" (UID: \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"
Sep 19 10:40:51.669772 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.669717    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/906df1aa-114e-4d1b-ac8d-3575e3c433d1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k\" (UID: \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"
Sep 19 10:40:51.669835 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.669782    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svk2j\" (UniqueName: \"kubernetes.io/projected/906df1aa-114e-4d1b-ac8d-3575e3c433d1-kube-api-access-svk2j\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k\" (UID: \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"
Sep 19 10:40:51.744005 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.743972    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"]
Sep 19 10:40:51.746190 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.746169    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"
Sep 19 10:40:51.756252 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.756228    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"]
Sep 19 10:40:51.770493 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.770470    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svk2j\" (UniqueName: \"kubernetes.io/projected/906df1aa-114e-4d1b-ac8d-3575e3c433d1-kube-api-access-svk2j\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k\" (UID: \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"
Sep 19 10:40:51.770608 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.770554    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/906df1aa-114e-4d1b-ac8d-3575e3c433d1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k\" (UID: \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"
Sep 19 10:40:51.770608 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.770588    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b029df1c-b0d4-49d9-abdd-5faafeeec444-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2\" (UID: \"b029df1c-b0d4-49d9-abdd-5faafeeec444\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"
Sep 19 10:40:51.770723 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.770615    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlx62\" (UniqueName: \"kubernetes.io/projected/b029df1c-b0d4-49d9-abdd-5faafeeec444-kube-api-access-hlx62\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2\" (UID: \"b029df1c-b0d4-49d9-abdd-5faafeeec444\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"
Sep 19 10:40:51.770778 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.770720    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/906df1aa-114e-4d1b-ac8d-3575e3c433d1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k\" (UID: \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"
Sep 19 10:40:51.770778 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.770759    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b029df1c-b0d4-49d9-abdd-5faafeeec444-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2\" (UID: \"b029df1c-b0d4-49d9-abdd-5faafeeec444\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"
Sep 19 10:40:51.770910 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.770894    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/906df1aa-114e-4d1b-ac8d-3575e3c433d1-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k\" (UID: \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"
Sep 19 10:40:51.770985 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.770970    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/906df1aa-114e-4d1b-ac8d-3575e3c433d1-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k\" (UID: \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"
Sep 19 10:40:51.779808 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.779790    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svk2j\" (UniqueName: \"kubernetes.io/projected/906df1aa-114e-4d1b-ac8d-3575e3c433d1-kube-api-access-svk2j\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k\" (UID: \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"
Sep 19 10:40:51.850297 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.850241    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"]
Sep 19 10:40:51.852514 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.852500    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"
Sep 19 10:40:51.870119 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.870094    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"]
Sep 19 10:40:51.871490 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.871472    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"
Sep 19 10:40:51.871674 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.871647    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b029df1c-b0d4-49d9-abdd-5faafeeec444-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2\" (UID: \"b029df1c-b0d4-49d9-abdd-5faafeeec444\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"
Sep 19 10:40:51.871780 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.871687    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlx62\" (UniqueName: \"kubernetes.io/projected/b029df1c-b0d4-49d9-abdd-5faafeeec444-kube-api-access-hlx62\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2\" (UID: \"b029df1c-b0d4-49d9-abdd-5faafeeec444\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"
Sep 19 10:40:51.871780 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.871744    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b029df1c-b0d4-49d9-abdd-5faafeeec444-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2\" (UID: \"b029df1c-b0d4-49d9-abdd-5faafeeec444\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"
Sep 19 10:40:51.871780 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.871775    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds7tj\" (UniqueName: \"kubernetes.io/projected/52e4e1e0-f78e-48d0-92f7-575206377990-kube-api-access-ds7tj\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5\" (UID: \"52e4e1e0-f78e-48d0-92f7-575206377990\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"
Sep 19 10:40:51.871943 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.871831    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e4e1e0-f78e-48d0-92f7-575206377990-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5\" (UID: \"52e4e1e0-f78e-48d0-92f7-575206377990\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"
Sep 19 10:40:51.871943 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.871861    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e4e1e0-f78e-48d0-92f7-575206377990-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5\" (UID: \"52e4e1e0-f78e-48d0-92f7-575206377990\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"
Sep 19 10:40:51.872083 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.872063    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b029df1c-b0d4-49d9-abdd-5faafeeec444-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2\" (UID: \"b029df1c-b0d4-49d9-abdd-5faafeeec444\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"
Sep 19 10:40:51.872178 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.872161    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b029df1c-b0d4-49d9-abdd-5faafeeec444-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2\" (UID: \"b029df1c-b0d4-49d9-abdd-5faafeeec444\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"
Sep 19 10:40:51.888308 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.888283    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlx62\" (UniqueName: \"kubernetes.io/projected/b029df1c-b0d4-49d9-abdd-5faafeeec444-kube-api-access-hlx62\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2\" (UID: \"b029df1c-b0d4-49d9-abdd-5faafeeec444\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"
Sep 19 10:40:51.955946 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.955911    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"
Sep 19 10:40:51.973185 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.973158    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds7tj\" (UniqueName: \"kubernetes.io/projected/52e4e1e0-f78e-48d0-92f7-575206377990-kube-api-access-ds7tj\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5\" (UID: \"52e4e1e0-f78e-48d0-92f7-575206377990\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"
Sep 19 10:40:51.973343 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.973219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e4e1e0-f78e-48d0-92f7-575206377990-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5\" (UID: \"52e4e1e0-f78e-48d0-92f7-575206377990\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"
Sep 19 10:40:51.973343 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.973257    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e4e1e0-f78e-48d0-92f7-575206377990-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5\" (UID: \"52e4e1e0-f78e-48d0-92f7-575206377990\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"
Sep 19 10:40:51.973343 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.973310    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5\" (UID: \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"
Sep 19 10:40:51.973498 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.973339    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzwkx\" (UniqueName: \"kubernetes.io/projected/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-kube-api-access-mzwkx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5\" (UID: \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"
Sep 19 10:40:51.973498 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.973375    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5\" (UID: \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"
Sep 19 10:40:51.973709 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.973689    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e4e1e0-f78e-48d0-92f7-575206377990-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5\" (UID: \"52e4e1e0-f78e-48d0-92f7-575206377990\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"
Sep 19 10:40:51.973709 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.973700    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e4e1e0-f78e-48d0-92f7-575206377990-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5\" (UID: \"52e4e1e0-f78e-48d0-92f7-575206377990\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"
Sep 19 10:40:51.985666 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:51.985641    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds7tj\" (UniqueName: \"kubernetes.io/projected/52e4e1e0-f78e-48d0-92f7-575206377990-kube-api-access-ds7tj\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5\" (UID: \"52e4e1e0-f78e-48d0-92f7-575206377990\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"
Sep 19 10:40:52.055152 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.055125    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"
Sep 19 10:40:52.073782 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.073756    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5\" (UID: \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"
Sep 19 10:40:52.073928 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.073798    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzwkx\" (UniqueName: \"kubernetes.io/projected/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-kube-api-access-mzwkx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5\" (UID: \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"
Sep 19 10:40:52.073928 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.073844    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5\" (UID: \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"
Sep 19 10:40:52.074138 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.074117    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5\" (UID: \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"
Sep 19 10:40:52.074241 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.074222    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5\" (UID: \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"
Sep 19 10:40:52.082830 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.082809    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzwkx\" (UniqueName: \"kubernetes.io/projected/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-kube-api-access-mzwkx\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5\" (UID: \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"
Sep 19 10:40:52.142605 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.142576    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"]
Sep 19 10:40:52.144044 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:40:52.144015    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906df1aa_114e_4d1b_ac8d_3575e3c433d1.slice/crio-b093ed6c573e0ffec9f5ba7df512dea8937bb158606106f5bd9dddbe9c28d091 WatchSource:0}: Error finding container b093ed6c573e0ffec9f5ba7df512dea8937bb158606106f5bd9dddbe9c28d091: Status 404 returned error can't find the container with id b093ed6c573e0ffec9f5ba7df512dea8937bb158606106f5bd9dddbe9c28d091
Sep 19 10:40:52.162298 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.162277    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"
Sep 19 10:40:52.241042 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.241001    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"]
Sep 19 10:40:52.244027 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:40:52.244001    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb029df1c_b0d4_49d9_abdd_5faafeeec444.slice/crio-6e94b356b38937752a42ff883832bb7dbc8ff894bb9511ac8d419586ce5d14ba WatchSource:0}: Error finding container 6e94b356b38937752a42ff883832bb7dbc8ff894bb9511ac8d419586ce5d14ba: Status 404 returned error can't find the container with id 6e94b356b38937752a42ff883832bb7dbc8ff894bb9511ac8d419586ce5d14ba
Sep 19 10:40:52.337548 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.337509    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"]
Sep 19 10:40:52.338293 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:40:52.338268    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52e4e1e0_f78e_48d0_92f7_575206377990.slice/crio-c1690482dcc4360f2c6864b0968c6a0b56c6075aef9156224115fd7574cf6ebd WatchSource:0}: Error finding container c1690482dcc4360f2c6864b0968c6a0b56c6075aef9156224115fd7574cf6ebd: Status 404 returned error can't find the container with id c1690482dcc4360f2c6864b0968c6a0b56c6075aef9156224115fd7574cf6ebd
Sep 19 10:40:52.439466 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.439436    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"]
Sep 19 10:40:52.441353 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:40:52.441321    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69ab3e5_f208_4d2a_8c4c_8fe91a92f6ca.slice/crio-14588a8e2202057bb390b79218b90ff9f14e71213fafd4079b95ac1d916c1a46 WatchSource:0}: Error finding container 14588a8e2202057bb390b79218b90ff9f14e71213fafd4079b95ac1d916c1a46: Status 404 returned error can't find the container with id 14588a8e2202057bb390b79218b90ff9f14e71213fafd4079b95ac1d916c1a46
Sep 19 10:40:52.706157 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.706126    2579 generic.go:356] "Generic (PLEG): container finished" podID="d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca" containerID="5e6cc270cfc88816a7382bd95e5256b704edb0274c37d12e1f925ffa7e96dace" exitCode=0
Sep 19 10:40:52.706518 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.706215    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5" event={"ID":"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca","Type":"ContainerDied","Data":"5e6cc270cfc88816a7382bd95e5256b704edb0274c37d12e1f925ffa7e96dace"}
Sep 19 10:40:52.706518 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.706270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5" event={"ID":"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca","Type":"ContainerStarted","Data":"14588a8e2202057bb390b79218b90ff9f14e71213fafd4079b95ac1d916c1a46"}
Sep 19 10:40:52.707214 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.707198    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 10:40:52.707861 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.707840    2579 generic.go:356] "Generic (PLEG): container finished" podID="b029df1c-b0d4-49d9-abdd-5faafeeec444" containerID="aa25f8c6b10118d26236b05e568376e176fb08029a6842142e2c5ae710be28f7" exitCode=0
Sep 19 10:40:52.707951 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.707928    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2" event={"ID":"b029df1c-b0d4-49d9-abdd-5faafeeec444","Type":"ContainerDied","Data":"aa25f8c6b10118d26236b05e568376e176fb08029a6842142e2c5ae710be28f7"}
Sep 19 10:40:52.708006 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.707978    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2" event={"ID":"b029df1c-b0d4-49d9-abdd-5faafeeec444","Type":"ContainerStarted","Data":"6e94b356b38937752a42ff883832bb7dbc8ff894bb9511ac8d419586ce5d14ba"}
Sep 19 10:40:52.709512 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.709485    2579 generic.go:356] "Generic (PLEG): container finished" podID="906df1aa-114e-4d1b-ac8d-3575e3c433d1" containerID="04268a36b09c8161bd16bf1ee61da8a84843b16a799ae919a426d9e072c7741f" exitCode=0
Sep 19 10:40:52.709622 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.709571    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k" event={"ID":"906df1aa-114e-4d1b-ac8d-3575e3c433d1","Type":"ContainerDied","Data":"04268a36b09c8161bd16bf1ee61da8a84843b16a799ae919a426d9e072c7741f"}
Sep 19 10:40:52.709681 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.709652    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k" event={"ID":"906df1aa-114e-4d1b-ac8d-3575e3c433d1","Type":"ContainerStarted","Data":"b093ed6c573e0ffec9f5ba7df512dea8937bb158606106f5bd9dddbe9c28d091"}
Sep 19 10:40:52.711288 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.711273    2579 generic.go:356] "Generic (PLEG): container finished" podID="52e4e1e0-f78e-48d0-92f7-575206377990" containerID="b18007403bec9b570218aba817d5c9d32ee67935f47cfc7fe87fa1b91bb632d2" exitCode=0
Sep 19 10:40:52.711353 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.711305    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5" event={"ID":"52e4e1e0-f78e-48d0-92f7-575206377990","Type":"ContainerDied","Data":"b18007403bec9b570218aba817d5c9d32ee67935f47cfc7fe87fa1b91bb632d2"}
Sep 19 10:40:52.711353 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:52.711334    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5" event={"ID":"52e4e1e0-f78e-48d0-92f7-575206377990","Type":"ContainerStarted","Data":"c1690482dcc4360f2c6864b0968c6a0b56c6075aef9156224115fd7574cf6ebd"}
Sep 19 10:40:53.717538 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:53.717497    2579 generic.go:356] "Generic (PLEG): container finished" podID="b029df1c-b0d4-49d9-abdd-5faafeeec444" containerID="e921962a491c71e6cd1dcfb35b06811ea38179f38b557e02ad0afd5c99306b25" exitCode=0
Sep 19 10:40:53.717905 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:53.717580    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2" event={"ID":"b029df1c-b0d4-49d9-abdd-5faafeeec444","Type":"ContainerDied","Data":"e921962a491c71e6cd1dcfb35b06811ea38179f38b557e02ad0afd5c99306b25"}
Sep 19 10:40:53.719592 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:53.719511    2579 generic.go:356] "Generic (PLEG): container finished" podID="906df1aa-114e-4d1b-ac8d-3575e3c433d1" containerID="d157d4022e03f4e6ca1b669b8cd3c29d2bdba88ee86760f78ea5b910a17b2685" exitCode=0
Sep 19 10:40:53.719684 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:53.719565    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k" event={"ID":"906df1aa-114e-4d1b-ac8d-3575e3c433d1","Type":"ContainerDied","Data":"d157d4022e03f4e6ca1b669b8cd3c29d2bdba88ee86760f78ea5b910a17b2685"}
Sep 19 10:40:53.721514 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:53.721497    2579 generic.go:356] "Generic (PLEG): container finished" podID="52e4e1e0-f78e-48d0-92f7-575206377990" containerID="7f393576539d9e11dd9ddc65d86f285e7cd4ffc601bc91ba4401ca22ac85925b" exitCode=0
Sep 19 10:40:53.721633 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:53.721562    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5" event={"ID":"52e4e1e0-f78e-48d0-92f7-575206377990","Type":"ContainerDied","Data":"7f393576539d9e11dd9ddc65d86f285e7cd4ffc601bc91ba4401ca22ac85925b"}
Sep 19 10:40:53.723471 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:53.723452    2579 generic.go:356] "Generic (PLEG): container finished" podID="d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca" containerID="c2b44570d9bbfef1afa13df9de29353422f4dd524225c19e62ddac86bfcb2a99" exitCode=0
Sep 19 10:40:53.723570 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:53.723479    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5" event={"ID":"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca","Type":"ContainerDied","Data":"c2b44570d9bbfef1afa13df9de29353422f4dd524225c19e62ddac86bfcb2a99"}
Sep 19 10:40:54.734356 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:54.734321    2579 generic.go:356] "Generic (PLEG): container finished" podID="d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca" containerID="fbb4451d563ba0315dd3ff1e8b6a15859d88f982529668221c2a29eeeea5a7df" exitCode=0
Sep 19 10:40:54.734743 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:54.734415    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5" event={"ID":"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca","Type":"ContainerDied","Data":"fbb4451d563ba0315dd3ff1e8b6a15859d88f982529668221c2a29eeeea5a7df"}
Sep 19 10:40:54.736599 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:54.736574    2579 generic.go:356] "Generic (PLEG): container finished" podID="b029df1c-b0d4-49d9-abdd-5faafeeec444" containerID="67550f4b0c368b632f47be9039fadcc6aea4f007817bacce4a815ed0cad97fd8" exitCode=0
Sep 19 10:40:54.736702 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:54.736659    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2" event={"ID":"b029df1c-b0d4-49d9-abdd-5faafeeec444","Type":"ContainerDied","Data":"67550f4b0c368b632f47be9039fadcc6aea4f007817bacce4a815ed0cad97fd8"}
Sep 19 10:40:54.738624 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:54.738608    2579 generic.go:356] "Generic (PLEG): container finished" podID="906df1aa-114e-4d1b-ac8d-3575e3c433d1" containerID="0cc4727613cf5a16d6a3c483967f8f034121ece9a261bbdbf81ab57836764a1f" exitCode=0
Sep 19 10:40:54.738701 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:54.738671    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k" event={"ID":"906df1aa-114e-4d1b-ac8d-3575e3c433d1","Type":"ContainerDied","Data":"0cc4727613cf5a16d6a3c483967f8f034121ece9a261bbdbf81ab57836764a1f"}
Sep 19 10:40:54.740581 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:54.740561    2579 generic.go:356] "Generic (PLEG): container finished" podID="52e4e1e0-f78e-48d0-92f7-575206377990" containerID="be6a33b067f56104532209f8707f14f81f57a42b1db5c296d48955858308e9f5" exitCode=0
Sep 19 10:40:54.740651 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:54.740620    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5" event={"ID":"52e4e1e0-f78e-48d0-92f7-575206377990","Type":"ContainerDied","Data":"be6a33b067f56104532209f8707f14f81f57a42b1db5c296d48955858308e9f5"}
Sep 19 10:40:55.882342 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:55.882321    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"
Sep 19 10:40:55.920486 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:55.920460    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"
Sep 19 10:40:55.950148 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:55.950126    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"
Sep 19 10:40:55.959222 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:55.959198    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"
Sep 19 10:40:56.003243 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.003183    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/906df1aa-114e-4d1b-ac8d-3575e3c433d1-util\") pod \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\" (UID: \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\") "
Sep 19 10:40:56.003243 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.003226    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzwkx\" (UniqueName: \"kubernetes.io/projected/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-kube-api-access-mzwkx\") pod \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\" (UID: \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\") "
Sep 19 10:40:56.003392 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.003269    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svk2j\" (UniqueName: \"kubernetes.io/projected/906df1aa-114e-4d1b-ac8d-3575e3c433d1-kube-api-access-svk2j\") pod \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\" (UID: \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\") "
Sep 19 10:40:56.003392 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.003287    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/906df1aa-114e-4d1b-ac8d-3575e3c433d1-bundle\") pod \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\" (UID: \"906df1aa-114e-4d1b-ac8d-3575e3c433d1\") "
Sep 19 10:40:56.003392 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.003305    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-util\") pod \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\" (UID: \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\") "
Sep 19 10:40:56.003392 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.003334    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-bundle\") pod \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\" (UID: \"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca\") "
Sep 19 10:40:56.003899 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.003874    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/906df1aa-114e-4d1b-ac8d-3575e3c433d1-bundle" (OuterVolumeSpecName: "bundle") pod "906df1aa-114e-4d1b-ac8d-3575e3c433d1" (UID: "906df1aa-114e-4d1b-ac8d-3575e3c433d1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:40:56.004009 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.003964    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-bundle" (OuterVolumeSpecName: "bundle") pod "d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca" (UID: "d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:40:56.005402 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.005383    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-kube-api-access-mzwkx" (OuterVolumeSpecName: "kube-api-access-mzwkx") pod "d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca" (UID: "d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca"). InnerVolumeSpecName "kube-api-access-mzwkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:40:56.005760 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.005737    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/906df1aa-114e-4d1b-ac8d-3575e3c433d1-kube-api-access-svk2j" (OuterVolumeSpecName: "kube-api-access-svk2j") pod "906df1aa-114e-4d1b-ac8d-3575e3c433d1" (UID: "906df1aa-114e-4d1b-ac8d-3575e3c433d1"). InnerVolumeSpecName "kube-api-access-svk2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:40:56.009263 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.009229    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-util" (OuterVolumeSpecName: "util") pod "d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca" (UID: "d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:40:56.009436 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.009419    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/906df1aa-114e-4d1b-ac8d-3575e3c433d1-util" (OuterVolumeSpecName: "util") pod "906df1aa-114e-4d1b-ac8d-3575e3c433d1" (UID: "906df1aa-114e-4d1b-ac8d-3575e3c433d1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:40:56.104575 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.104547    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e4e1e0-f78e-48d0-92f7-575206377990-bundle\") pod \"52e4e1e0-f78e-48d0-92f7-575206377990\" (UID: \"52e4e1e0-f78e-48d0-92f7-575206377990\") "
Sep 19 10:40:56.104694 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.104583    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b029df1c-b0d4-49d9-abdd-5faafeeec444-util\") pod \"b029df1c-b0d4-49d9-abdd-5faafeeec444\" (UID: \"b029df1c-b0d4-49d9-abdd-5faafeeec444\") "
Sep 19 10:40:56.104694 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.104635    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlx62\" (UniqueName: \"kubernetes.io/projected/b029df1c-b0d4-49d9-abdd-5faafeeec444-kube-api-access-hlx62\") pod \"b029df1c-b0d4-49d9-abdd-5faafeeec444\" (UID: \"b029df1c-b0d4-49d9-abdd-5faafeeec444\") "
Sep 19 10:40:56.104694 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.104677    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b029df1c-b0d4-49d9-abdd-5faafeeec444-bundle\") pod \"b029df1c-b0d4-49d9-abdd-5faafeeec444\" (UID: \"b029df1c-b0d4-49d9-abdd-5faafeeec444\") "
Sep 19 10:40:56.104844 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.104712    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ds7tj\" (UniqueName: \"kubernetes.io/projected/52e4e1e0-f78e-48d0-92f7-575206377990-kube-api-access-ds7tj\") pod \"52e4e1e0-f78e-48d0-92f7-575206377990\" (UID: \"52e4e1e0-f78e-48d0-92f7-575206377990\") "
Sep 19 10:40:56.104844 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.104755    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e4e1e0-f78e-48d0-92f7-575206377990-util\") pod \"52e4e1e0-f78e-48d0-92f7-575206377990\" (UID: \"52e4e1e0-f78e-48d0-92f7-575206377990\") "
Sep 19 10:40:56.104980 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.104962    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svk2j\" (UniqueName: \"kubernetes.io/projected/906df1aa-114e-4d1b-ac8d-3575e3c433d1-kube-api-access-svk2j\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:40:56.105043 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.104987    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/906df1aa-114e-4d1b-ac8d-3575e3c433d1-bundle\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:40:56.105043 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.105000    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-util\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:40:56.105043 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.105013    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-bundle\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:40:56.105043 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.105025    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/906df1aa-114e-4d1b-ac8d-3575e3c433d1-util\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:40:56.105043 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.105037    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzwkx\" (UniqueName: \"kubernetes.io/projected/d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca-kube-api-access-mzwkx\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:40:56.105253 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.105233    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52e4e1e0-f78e-48d0-92f7-575206377990-bundle" (OuterVolumeSpecName: "bundle") pod "52e4e1e0-f78e-48d0-92f7-575206377990" (UID: "52e4e1e0-f78e-48d0-92f7-575206377990"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:40:56.105440 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.105420    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b029df1c-b0d4-49d9-abdd-5faafeeec444-bundle" (OuterVolumeSpecName: "bundle") pod "b029df1c-b0d4-49d9-abdd-5faafeeec444" (UID: "b029df1c-b0d4-49d9-abdd-5faafeeec444"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:40:56.106692 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.106672    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b029df1c-b0d4-49d9-abdd-5faafeeec444-kube-api-access-hlx62" (OuterVolumeSpecName: "kube-api-access-hlx62") pod "b029df1c-b0d4-49d9-abdd-5faafeeec444" (UID: "b029df1c-b0d4-49d9-abdd-5faafeeec444"). InnerVolumeSpecName "kube-api-access-hlx62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:40:56.106777 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.106738    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52e4e1e0-f78e-48d0-92f7-575206377990-kube-api-access-ds7tj" (OuterVolumeSpecName: "kube-api-access-ds7tj") pod "52e4e1e0-f78e-48d0-92f7-575206377990" (UID: "52e4e1e0-f78e-48d0-92f7-575206377990"). InnerVolumeSpecName "kube-api-access-ds7tj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:40:56.113457 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.113440    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b029df1c-b0d4-49d9-abdd-5faafeeec444-util" (OuterVolumeSpecName: "util") pod "b029df1c-b0d4-49d9-abdd-5faafeeec444" (UID: "b029df1c-b0d4-49d9-abdd-5faafeeec444"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:40:56.113723 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.113705    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52e4e1e0-f78e-48d0-92f7-575206377990-util" (OuterVolumeSpecName: "util") pod "52e4e1e0-f78e-48d0-92f7-575206377990" (UID: "52e4e1e0-f78e-48d0-92f7-575206377990"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:40:56.205691 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.205660    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds7tj\" (UniqueName: \"kubernetes.io/projected/52e4e1e0-f78e-48d0-92f7-575206377990-kube-api-access-ds7tj\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:40:56.205691 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.205689    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e4e1e0-f78e-48d0-92f7-575206377990-util\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:40:56.205821 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.205772    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e4e1e0-f78e-48d0-92f7-575206377990-bundle\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:40:56.205821 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.205781    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b029df1c-b0d4-49d9-abdd-5faafeeec444-util\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:40:56.205821 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.205789    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlx62\" (UniqueName: \"kubernetes.io/projected/b029df1c-b0d4-49d9-abdd-5faafeeec444-kube-api-access-hlx62\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:40:56.205821 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.205798    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b029df1c-b0d4-49d9-abdd-5faafeeec444-bundle\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:40:56.753063 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.753034    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5"
Sep 19 10:40:56.753258 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.753069    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6krsh5" event={"ID":"52e4e1e0-f78e-48d0-92f7-575206377990","Type":"ContainerDied","Data":"c1690482dcc4360f2c6864b0968c6a0b56c6075aef9156224115fd7574cf6ebd"}
Sep 19 10:40:56.753258 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.753113    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1690482dcc4360f2c6864b0968c6a0b56c6075aef9156224115fd7574cf6ebd"
Sep 19 10:40:56.754848 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.754830    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5"
Sep 19 10:40:56.754944 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.754834    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79qlr5" event={"ID":"d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca","Type":"ContainerDied","Data":"14588a8e2202057bb390b79218b90ff9f14e71213fafd4079b95ac1d916c1a46"}
Sep 19 10:40:56.755001 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.754949    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14588a8e2202057bb390b79218b90ff9f14e71213fafd4079b95ac1d916c1a46"
Sep 19 10:40:56.756792 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.756777    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2"
Sep 19 10:40:56.756863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.756775    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azkps2" event={"ID":"b029df1c-b0d4-49d9-abdd-5faafeeec444","Type":"ContainerDied","Data":"6e94b356b38937752a42ff883832bb7dbc8ff894bb9511ac8d419586ce5d14ba"}
Sep 19 10:40:56.756905 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.756884    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e94b356b38937752a42ff883832bb7dbc8ff894bb9511ac8d419586ce5d14ba"
Sep 19 10:40:56.758637 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.758611    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k" event={"ID":"906df1aa-114e-4d1b-ac8d-3575e3c433d1","Type":"ContainerDied","Data":"b093ed6c573e0ffec9f5ba7df512dea8937bb158606106f5bd9dddbe9c28d091"}
Sep 19 10:40:56.758711 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.758645    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cpnc7k"
Sep 19 10:40:56.758711 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:40:56.758650    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b093ed6c573e0ffec9f5ba7df512dea8937bb158606106f5bd9dddbe9c28d091"
Sep 19 10:41:00.143155 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:00.143116    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:00.254610 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:00.254578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:00.939126 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:00.939084    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:01.756652 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:01.756622    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:04.972365 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:04.972330    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:05.075249 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:05.075218    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:05.360560 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:05.360413    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:07.410009 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:07.409979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:07.576043 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:07.576002    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:08.598614 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:08.598567    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:09.359655 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:09.359615    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:09.748253 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:09.748219    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:10.805579 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:10.805513    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:12.451696 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.451657    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw"]
Sep 19 10:41:12.452042 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.451966    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca" containerName="pull"
Sep 19 10:41:12.452042 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.451979    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca" containerName="pull"
Sep 19 10:41:12.452042 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.451991    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906df1aa-114e-4d1b-ac8d-3575e3c433d1" containerName="extract"
Sep 19 10:41:12.452042 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.451997    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="906df1aa-114e-4d1b-ac8d-3575e3c433d1" containerName="extract"
Sep 19 10:41:12.452042 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452009    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b029df1c-b0d4-49d9-abdd-5faafeeec444" containerName="util"
Sep 19 10:41:12.452042 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452014    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b029df1c-b0d4-49d9-abdd-5faafeeec444" containerName="util"
Sep 19 10:41:12.452042 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452021    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906df1aa-114e-4d1b-ac8d-3575e3c433d1" containerName="pull"
Sep 19 10:41:12.452042 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452025    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="906df1aa-114e-4d1b-ac8d-3575e3c433d1" containerName="pull"
Sep 19 10:41:12.452042 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452031    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52e4e1e0-f78e-48d0-92f7-575206377990" containerName="extract"
Sep 19 10:41:12.452042 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452039    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="52e4e1e0-f78e-48d0-92f7-575206377990" containerName="extract"
Sep 19 10:41:12.452042 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452048    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca" containerName="extract"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452053    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca" containerName="extract"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452060    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca" containerName="util"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452065    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca" containerName="util"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452072    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b029df1c-b0d4-49d9-abdd-5faafeeec444" containerName="pull"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452077    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b029df1c-b0d4-49d9-abdd-5faafeeec444" containerName="pull"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452082    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b029df1c-b0d4-49d9-abdd-5faafeeec444" containerName="extract"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452087    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b029df1c-b0d4-49d9-abdd-5faafeeec444" containerName="extract"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452098    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52e4e1e0-f78e-48d0-92f7-575206377990" containerName="util"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452102    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="52e4e1e0-f78e-48d0-92f7-575206377990" containerName="util"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452108    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52e4e1e0-f78e-48d0-92f7-575206377990" containerName="pull"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452116    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="52e4e1e0-f78e-48d0-92f7-575206377990" containerName="pull"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452126    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906df1aa-114e-4d1b-ac8d-3575e3c433d1" containerName="util"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452131    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="906df1aa-114e-4d1b-ac8d-3575e3c433d1" containerName="util"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452174    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="906df1aa-114e-4d1b-ac8d-3575e3c433d1" containerName="extract"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452183    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="52e4e1e0-f78e-48d0-92f7-575206377990" containerName="extract"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452190    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d69ab3e5-f208-4d2a-8c4c-8fe91a92f6ca" containerName="extract"
Sep 19 10:41:12.452355 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.452200    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b029df1c-b0d4-49d9-abdd-5faafeeec444" containerName="extract"
Sep 19 10:41:12.456292 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.456276    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw"
Sep 19 10:41:12.460046 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.460027    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 19 10:41:12.460176 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.460071    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 19 10:41:12.461151 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.461127    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 19 10:41:12.461151 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.461131    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mwjf4\""
Sep 19 10:41:12.476116 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.476097    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:12.482035 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.482019    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 19 10:41:12.484334 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.484313    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw"]
Sep 19 10:41:12.634672 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.634640    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cflz7\" (UniqueName: \"kubernetes.io/projected/add13f26-9889-4f49-b89d-df992594a021-kube-api-access-cflz7\") pod \"kuadrant-console-plugin-fbdd4f445-lkvnw\" (UID: \"add13f26-9889-4f49-b89d-df992594a021\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw"
Sep 19 10:41:12.634827 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.634686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/add13f26-9889-4f49-b89d-df992594a021-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-lkvnw\" (UID: \"add13f26-9889-4f49-b89d-df992594a021\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw"
Sep 19 10:41:12.634827 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.634754    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/add13f26-9889-4f49-b89d-df992594a021-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-lkvnw\" (UID: \"add13f26-9889-4f49-b89d-df992594a021\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw"
Sep 19 10:41:12.735310 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.735244    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/add13f26-9889-4f49-b89d-df992594a021-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-lkvnw\" (UID: \"add13f26-9889-4f49-b89d-df992594a021\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw"
Sep 19 10:41:12.735310 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.735297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cflz7\" (UniqueName: \"kubernetes.io/projected/add13f26-9889-4f49-b89d-df992594a021-kube-api-access-cflz7\") pod \"kuadrant-console-plugin-fbdd4f445-lkvnw\" (UID: \"add13f26-9889-4f49-b89d-df992594a021\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw"
Sep 19 10:41:12.735455 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.735328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/add13f26-9889-4f49-b89d-df992594a021-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-lkvnw\" (UID: \"add13f26-9889-4f49-b89d-df992594a021\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw"
Sep 19 10:41:12.735937 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.735918    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/add13f26-9889-4f49-b89d-df992594a021-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-lkvnw\" (UID: \"add13f26-9889-4f49-b89d-df992594a021\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw"
Sep 19 10:41:12.737650 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.737629    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/add13f26-9889-4f49-b89d-df992594a021-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-lkvnw\" (UID: \"add13f26-9889-4f49-b89d-df992594a021\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw"
Sep 19 10:41:12.745461 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.745442    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cflz7\" (UniqueName: \"kubernetes.io/projected/add13f26-9889-4f49-b89d-df992594a021-kube-api-access-cflz7\") pod \"kuadrant-console-plugin-fbdd4f445-lkvnw\" (UID: \"add13f26-9889-4f49-b89d-df992594a021\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw"
Sep 19 10:41:12.765293 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.765272    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw"
Sep 19 10:41:12.775644 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:12.775622    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:13.046281 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:13.046077    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw"]
Sep 19 10:41:13.048774 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:41:13.048748    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadd13f26_9889_4f49_b89d_df992594a021.slice/crio-9b7ffeed3fcc08baf5851fd3667eda5461f0d19b273810e210e436a4d993ec20 WatchSource:0}: Error finding container 9b7ffeed3fcc08baf5851fd3667eda5461f0d19b273810e210e436a4d993ec20: Status 404 returned error can't find the container with id 9b7ffeed3fcc08baf5851fd3667eda5461f0d19b273810e210e436a4d993ec20
Sep 19 10:41:13.844682 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:13.844646    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw" event={"ID":"add13f26-9889-4f49-b89d-df992594a021","Type":"ContainerStarted","Data":"9b7ffeed3fcc08baf5851fd3667eda5461f0d19b273810e210e436a4d993ec20"}
Sep 19 10:41:15.355871 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:15.355838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:15.422366 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:15.422325    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:15.753684 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:15.753649    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:15.814275 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:15.814240    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:16.202781 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:16.202743    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:16.271347 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:16.271317    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:17.866899 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:17.866865    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw" event={"ID":"add13f26-9889-4f49-b89d-df992594a021","Type":"ContainerStarted","Data":"fa00a26eb1b9407b54f77c11f5bce330ee7c97a8b3a3e7958824376003c388ee"}
Sep 19 10:41:17.884910 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:17.884863    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-lkvnw" podStartSLOduration=1.546483146 podStartE2EDuration="5.884848091s" totalImagesPullingTime="4.338364945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:41:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:41:17.882970896 +0000 UTC m=+398.794664415" watchObservedRunningTime="2026-09-19 10:41:17.884848091 +0000 UTC m=+398.796541611"
Sep 19 10:41:19.074496 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:19.074457    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:19.174780 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:19.174745    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:19.276877 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:19.276849    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:19.812767 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:19.812729    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:19.879219 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:19.879180    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:22.443897 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:22.443855    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:22.548065 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:22.548015    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:24.942969 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:24.942935    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:25.120430 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:25.120398    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:30.220067 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:30.220031    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:30.341240 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:30.341212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:30.411246 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:30.411209    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:31.367481 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:31.367451    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:31.468213 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:31.468184    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:36.564788 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:36.564756    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:36.666634 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:36.666603    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:37.048735 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:37.048701    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:37.151797 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:37.151752    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:37.251442 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:37.251412    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:44.179325 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:44.179292    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:44.280934 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:44.280905    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:46.617561 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:46.617481    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:46.718071 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:46.718034    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:53.182897 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:53.182860    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:53.279607 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:53.279562    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:56.582585 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:56.582547    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:41:56.997740 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:41:56.997704    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:01.185497 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.185457    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:01.306484 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.306446    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:01.381434 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.381396    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:01.552403 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.552336    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:42:01.555510 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.555495    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"
Sep 19 10:42:01.557977 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.557954    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 19 10:42:01.571571 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.571546    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:42:01.581642 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.581618    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:42:01.721872 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.721840    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qdcd\" (UniqueName: \"kubernetes.io/projected/13ce7aab-198d-412a-9c20-c5f5ca6fac1e-kube-api-access-6qdcd\") pod \"limitador-limitador-84994f7ffc-6xxmd\" (UID: \"13ce7aab-198d-412a-9c20-c5f5ca6fac1e\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"
Sep 19 10:42:01.722043 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.721895    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/13ce7aab-198d-412a-9c20-c5f5ca6fac1e-config-file\") pod \"limitador-limitador-84994f7ffc-6xxmd\" (UID: \"13ce7aab-198d-412a-9c20-c5f5ca6fac1e\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"
Sep 19 10:42:01.823124 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.823037    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qdcd\" (UniqueName: \"kubernetes.io/projected/13ce7aab-198d-412a-9c20-c5f5ca6fac1e-kube-api-access-6qdcd\") pod \"limitador-limitador-84994f7ffc-6xxmd\" (UID: \"13ce7aab-198d-412a-9c20-c5f5ca6fac1e\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"
Sep 19 10:42:01.823124 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.823081    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/13ce7aab-198d-412a-9c20-c5f5ca6fac1e-config-file\") pod \"limitador-limitador-84994f7ffc-6xxmd\" (UID: \"13ce7aab-198d-412a-9c20-c5f5ca6fac1e\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"
Sep 19 10:42:01.823706 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.823688    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/13ce7aab-198d-412a-9c20-c5f5ca6fac1e-config-file\") pod \"limitador-limitador-84994f7ffc-6xxmd\" (UID: \"13ce7aab-198d-412a-9c20-c5f5ca6fac1e\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"
Sep 19 10:42:01.830740 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.830719    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qdcd\" (UniqueName: \"kubernetes.io/projected/13ce7aab-198d-412a-9c20-c5f5ca6fac1e-kube-api-access-6qdcd\") pod \"limitador-limitador-84994f7ffc-6xxmd\" (UID: \"13ce7aab-198d-412a-9c20-c5f5ca6fac1e\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"
Sep 19 10:42:01.865724 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:01.865692    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"
Sep 19 10:42:02.082280 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:02.082189    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:02.143555 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:02.143512    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:42:02.145271 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:42:02.145248    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ce7aab_198d_412a_9c20_c5f5ca6fac1e.slice/crio-48376f834de8cd2963007a787a37626ece715633c4de5d6dc21eec4771b57df3 WatchSource:0}: Error finding container 48376f834de8cd2963007a787a37626ece715633c4de5d6dc21eec4771b57df3: Status 404 returned error can't find the container with id 48376f834de8cd2963007a787a37626ece715633c4de5d6dc21eec4771b57df3
Sep 19 10:42:02.155790 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:02.155769    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-pxv9v"]
Sep 19 10:42:02.160110 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:02.160093    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-pxv9v"
Sep 19 10:42:02.161174 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:02.161150    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:02.162783 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:02.162763    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dq945\""
Sep 19 10:42:02.167243 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:02.167186    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pxv9v"]
Sep 19 10:42:02.327797 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:02.327754    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6klw7\" (UniqueName: \"kubernetes.io/projected/d225ebaa-6642-4252-951f-265b7229d385-kube-api-access-6klw7\") pod \"authorino-59c974bd48-pxv9v\" (UID: \"d225ebaa-6642-4252-951f-265b7229d385\") " pod="kuadrant-system/authorino-59c974bd48-pxv9v"
Sep 19 10:42:02.429119 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:02.429082    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6klw7\" (UniqueName: \"kubernetes.io/projected/d225ebaa-6642-4252-951f-265b7229d385-kube-api-access-6klw7\") pod \"authorino-59c974bd48-pxv9v\" (UID: \"d225ebaa-6642-4252-951f-265b7229d385\") " pod="kuadrant-system/authorino-59c974bd48-pxv9v"
Sep 19 10:42:02.437952 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:02.437927    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6klw7\" (UniqueName: \"kubernetes.io/projected/d225ebaa-6642-4252-951f-265b7229d385-kube-api-access-6klw7\") pod \"authorino-59c974bd48-pxv9v\" (UID: \"d225ebaa-6642-4252-951f-265b7229d385\") " pod="kuadrant-system/authorino-59c974bd48-pxv9v"
Sep 19 10:42:02.469977 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:02.469948    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-pxv9v"
Sep 19 10:42:02.740856 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:02.740828    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pxv9v"]
Sep 19 10:42:02.742370 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:42:02.742344    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd225ebaa_6642_4252_951f_265b7229d385.slice/crio-b0f89a51445d46ca045da653b284e256c177c84dbb6650860df14bce8733647b WatchSource:0}: Error finding container b0f89a51445d46ca045da653b284e256c177c84dbb6650860df14bce8733647b: Status 404 returned error can't find the container with id b0f89a51445d46ca045da653b284e256c177c84dbb6650860df14bce8733647b
Sep 19 10:42:03.082268 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:03.082180    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-pxv9v" event={"ID":"d225ebaa-6642-4252-951f-265b7229d385","Type":"ContainerStarted","Data":"b0f89a51445d46ca045da653b284e256c177c84dbb6650860df14bce8733647b"}
Sep 19 10:42:03.086003 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:03.085972    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-6xxmd" event={"ID":"13ce7aab-198d-412a-9c20-c5f5ca6fac1e","Type":"ContainerStarted","Data":"48376f834de8cd2963007a787a37626ece715633c4de5d6dc21eec4771b57df3"}
Sep 19 10:42:06.102741 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:06.102703    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-pxv9v" event={"ID":"d225ebaa-6642-4252-951f-265b7229d385","Type":"ContainerStarted","Data":"b56728e4b1cb98945238cff34c579ce3c3b803b79e33d913923794f290a8a14e"}
Sep 19 10:42:06.104251 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:06.104228    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-6xxmd" event={"ID":"13ce7aab-198d-412a-9c20-c5f5ca6fac1e","Type":"ContainerStarted","Data":"bffc9747b51675611478be456ef489cb3483294812ae2ad3ee7716ac120ee7aa"}
Sep 19 10:42:06.104359 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:06.104275    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"
Sep 19 10:42:06.118143 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:06.118102    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-pxv9v" podStartSLOduration=1.356096138 podStartE2EDuration="4.118088774s" totalImagesPullingTime="2.761992636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:42:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:42:06.117245283 +0000 UTC m=+447.028938802" watchObservedRunningTime="2026-09-19 10:42:06.118088774 +0000 UTC m=+447.029782292"
Sep 19 10:42:06.134926 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:06.134877    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-6xxmd" podStartSLOduration=1.7764039010000001 podStartE2EDuration="5.134863344s" totalImagesPullingTime="3.358459443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:42:06.134488244 +0000 UTC m=+447.046181763" watchObservedRunningTime="2026-09-19 10:42:06.134863344 +0000 UTC m=+447.046556864"
Sep 19 10:42:17.114401 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:17.114375    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"
Sep 19 10:42:22.386930 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:22.386898    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:22.488742 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:22.488712    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:22.900598 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:22.900569    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:34.693742 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:34.693712    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:34.792864 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:34.792834    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:36.438594 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:36.438559    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pxv9v"]
Sep 19 10:42:36.438952 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:36.438736    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-pxv9v" podUID="d225ebaa-6642-4252-951f-265b7229d385" containerName="authorino" containerID="cri-o://b56728e4b1cb98945238cff34c579ce3c3b803b79e33d913923794f290a8a14e" gracePeriod=30
Sep 19 10:42:36.675357 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:36.675332    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-pxv9v"
Sep 19 10:42:36.796886 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:36.796824    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6klw7\" (UniqueName: \"kubernetes.io/projected/d225ebaa-6642-4252-951f-265b7229d385-kube-api-access-6klw7\") pod \"d225ebaa-6642-4252-951f-265b7229d385\" (UID: \"d225ebaa-6642-4252-951f-265b7229d385\") "
Sep 19 10:42:36.798864 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:36.798830    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d225ebaa-6642-4252-951f-265b7229d385-kube-api-access-6klw7" (OuterVolumeSpecName: "kube-api-access-6klw7") pod "d225ebaa-6642-4252-951f-265b7229d385" (UID: "d225ebaa-6642-4252-951f-265b7229d385"). InnerVolumeSpecName "kube-api-access-6klw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:42:36.897855 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:36.897825    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6klw7\" (UniqueName: \"kubernetes.io/projected/d225ebaa-6642-4252-951f-265b7229d385-kube-api-access-6klw7\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:42:37.254626 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:37.254594    2579 generic.go:356] "Generic (PLEG): container finished" podID="d225ebaa-6642-4252-951f-265b7229d385" containerID="b56728e4b1cb98945238cff34c579ce3c3b803b79e33d913923794f290a8a14e" exitCode=0
Sep 19 10:42:37.254802 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:37.254640    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-pxv9v"
Sep 19 10:42:37.254802 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:37.254672    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-pxv9v" event={"ID":"d225ebaa-6642-4252-951f-265b7229d385","Type":"ContainerDied","Data":"b56728e4b1cb98945238cff34c579ce3c3b803b79e33d913923794f290a8a14e"}
Sep 19 10:42:37.254802 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:37.254711    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-pxv9v" event={"ID":"d225ebaa-6642-4252-951f-265b7229d385","Type":"ContainerDied","Data":"b0f89a51445d46ca045da653b284e256c177c84dbb6650860df14bce8733647b"}
Sep 19 10:42:37.254802 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:37.254727    2579 scope.go:117] "RemoveContainer" containerID="b56728e4b1cb98945238cff34c579ce3c3b803b79e33d913923794f290a8a14e"
Sep 19 10:42:37.262977 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:37.262961    2579 scope.go:117] "RemoveContainer" containerID="b56728e4b1cb98945238cff34c579ce3c3b803b79e33d913923794f290a8a14e"
Sep 19 10:42:37.263226 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:42:37.263207    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b56728e4b1cb98945238cff34c579ce3c3b803b79e33d913923794f290a8a14e\": container with ID starting with b56728e4b1cb98945238cff34c579ce3c3b803b79e33d913923794f290a8a14e not found: ID does not exist" containerID="b56728e4b1cb98945238cff34c579ce3c3b803b79e33d913923794f290a8a14e"
Sep 19 10:42:37.263290 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:37.263234    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b56728e4b1cb98945238cff34c579ce3c3b803b79e33d913923794f290a8a14e"} err="failed to get container status \"b56728e4b1cb98945238cff34c579ce3c3b803b79e33d913923794f290a8a14e\": rpc error: code = NotFound desc = could not find container \"b56728e4b1cb98945238cff34c579ce3c3b803b79e33d913923794f290a8a14e\": container with ID starting with b56728e4b1cb98945238cff34c579ce3c3b803b79e33d913923794f290a8a14e not found: ID does not exist"
Sep 19 10:42:37.273970 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:37.273945    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pxv9v"]
Sep 19 10:42:37.275805 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:37.275783    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-pxv9v"]
Sep 19 10:42:39.115083 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:39.115051    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d225ebaa-6642-4252-951f-265b7229d385" path="/var/lib/kubelet/pods/d225ebaa-6642-4252-951f-265b7229d385/volumes"
Sep 19 10:42:41.136190 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:41.136159    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:41.236169 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:41.236141    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:41.300393 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:41.300330    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:41.465605 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:41.465572    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:41.916858 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:41.916824    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:45.103396 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:45.103358    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:45.202927 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:45.202895    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:42:57.897901 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:42:57.897868    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:43:16.891993 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:16.891952    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:43:16.990772 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:16.990737    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:43:18.407693 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:18.407603    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:43:18.489881 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:18.489845    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:43:18.590752 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:18.590721    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:43:18.885422 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:18.885391    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:43:18.986337 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:18.986310    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:43:19.074161 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:19.074131    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:43:19.092331 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:19.092307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:43:29.657904 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:29.657833    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:43:29.741823 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:29.741782    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:43:34.429182 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:34.429153    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:43:46.087896 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:46.087855    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:43:46.452673 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:46.452643    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:43:51.344738 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:51.344705    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:43:51.916836 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:43:51.916799    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:44:08.038697 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.038668    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"]
Sep 19 10:44:08.039037 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.038959    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d225ebaa-6642-4252-951f-265b7229d385" containerName="authorino"
Sep 19 10:44:08.039037 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.038970    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d225ebaa-6642-4252-951f-265b7229d385" containerName="authorino"
Sep 19 10:44:08.039107 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.039043    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d225ebaa-6642-4252-951f-265b7229d385" containerName="authorino"
Sep 19 10:44:08.041897 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.041882    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.044730 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.044708    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 19 10:44:08.044730 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.044728    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 19 10:44:08.046190 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.046170    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4rnbf\""
Sep 19 10:44:08.046301 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.046204    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 19 10:44:08.056283 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.056253    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"]
Sep 19 10:44:08.074645 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.074617    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:44:08.095236 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.095210    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.095320 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.095240    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.095320 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.095261    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldgzq\" (UniqueName: \"kubernetes.io/projected/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-kube-api-access-ldgzq\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.095395 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.095335    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.095395 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.095357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.095395 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.095377    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.196586 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.196547    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.196702 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.196618    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.196702 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.196648    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.196702 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.196692    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldgzq\" (UniqueName: \"kubernetes.io/projected/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-kube-api-access-ldgzq\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.196858 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.196751    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.196858 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.196777    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.197030 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.197009    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.197153 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.197117    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.197230 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.197195    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.198898 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.198879    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.199043 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.199026    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.208433 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.208413    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldgzq\" (UniqueName: \"kubernetes.io/projected/5f6db6b8-47c6-44f9-8c29-f3a203ba192f-kube-api-access-ldgzq\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh\" (UID: \"5f6db6b8-47c6-44f9-8c29-f3a203ba192f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.358770 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.358688    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:08.630534 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.630496    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"]
Sep 19 10:44:08.632583 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:44:08.632553    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f6db6b8_47c6_44f9_8c29_f3a203ba192f.slice/crio-53d5ad6d5b95bce4ebfbcc1d3bf8b54872ec2ceefbb4d3bce4bc3c8ef8f15575 WatchSource:0}: Error finding container 53d5ad6d5b95bce4ebfbcc1d3bf8b54872ec2ceefbb4d3bce4bc3c8ef8f15575: Status 404 returned error can't find the container with id 53d5ad6d5b95bce4ebfbcc1d3bf8b54872ec2ceefbb4d3bce4bc3c8ef8f15575
Sep 19 10:44:08.691811 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.691775    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh" event={"ID":"5f6db6b8-47c6-44f9-8c29-f3a203ba192f","Type":"ContainerStarted","Data":"53d5ad6d5b95bce4ebfbcc1d3bf8b54872ec2ceefbb4d3bce4bc3c8ef8f15575"}
Sep 19 10:44:08.898974 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:08.898884    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:44:09.698375 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:09.698344    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh" event={"ID":"5f6db6b8-47c6-44f9-8c29-f3a203ba192f","Type":"ContainerStarted","Data":"b1086cae27d2f2518c4e75b32b39c0b34570bf43c97f6713561149283cfef176"}
Sep 19 10:44:09.718702 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:09.718657    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh" podStartSLOduration=0.744367603 podStartE2EDuration="1.718642031s" totalImagesPullingTime="974.274428ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:44:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:44:09.716283805 +0000 UTC m=+570.627977327" watchObservedRunningTime="2026-09-19 10:44:09.718642031 +0000 UTC m=+570.630335547"
Sep 19 10:44:15.441426 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:15.441384    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:44:16.804931 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:16.804893    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:44:18.359696 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:18.359589    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:18.359696 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:18.359656    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:18.371840 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:18.371817    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:18.751107 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:18.751079    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh"
Sep 19 10:44:24.250735 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.250702    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"]
Sep 19 10:44:24.254263 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.254240    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.260764 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.260603    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 19 10:44:24.267791 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.267767    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"]
Sep 19 10:44:24.289208 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.289180    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:44:24.427342 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.427310    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60c115c6-2bfa-47eb-81ba-6620bb7239cc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.427491 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.427354    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60c115c6-2bfa-47eb-81ba-6620bb7239cc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.427491 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.427413    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60c115c6-2bfa-47eb-81ba-6620bb7239cc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.427491 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.427451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60c115c6-2bfa-47eb-81ba-6620bb7239cc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.427491 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.427472    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60c115c6-2bfa-47eb-81ba-6620bb7239cc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.427659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.427498    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66v55\" (UniqueName: \"kubernetes.io/projected/60c115c6-2bfa-47eb-81ba-6620bb7239cc-kube-api-access-66v55\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.528590 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.528492    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60c115c6-2bfa-47eb-81ba-6620bb7239cc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.528590 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.528550    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60c115c6-2bfa-47eb-81ba-6620bb7239cc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.528590 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.528574    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60c115c6-2bfa-47eb-81ba-6620bb7239cc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.528590 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.528594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60c115c6-2bfa-47eb-81ba-6620bb7239cc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.528894 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.528616    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66v55\" (UniqueName: \"kubernetes.io/projected/60c115c6-2bfa-47eb-81ba-6620bb7239cc-kube-api-access-66v55\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.528894 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.528657    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60c115c6-2bfa-47eb-81ba-6620bb7239cc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.528997 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.528933    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60c115c6-2bfa-47eb-81ba-6620bb7239cc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.528997 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.528967    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60c115c6-2bfa-47eb-81ba-6620bb7239cc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.529063 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.528999    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60c115c6-2bfa-47eb-81ba-6620bb7239cc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.530803 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.530783    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60c115c6-2bfa-47eb-81ba-6620bb7239cc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.530968 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.530951    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60c115c6-2bfa-47eb-81ba-6620bb7239cc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.537028 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.537007    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66v55\" (UniqueName: \"kubernetes.io/projected/60c115c6-2bfa-47eb-81ba-6620bb7239cc-kube-api-access-66v55\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6\" (UID: \"60c115c6-2bfa-47eb-81ba-6620bb7239cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.564463 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.564444    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:24.839780 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:24.839752    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"]
Sep 19 10:44:24.841339 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:44:24.841308    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60c115c6_2bfa_47eb_81ba_6620bb7239cc.slice/crio-3c9877629fe2a92ee22204e8a150ad229f06f88e81b330ab9826c4de8316a4c4 WatchSource:0}: Error finding container 3c9877629fe2a92ee22204e8a150ad229f06f88e81b330ab9826c4de8316a4c4: Status 404 returned error can't find the container with id 3c9877629fe2a92ee22204e8a150ad229f06f88e81b330ab9826c4de8316a4c4
Sep 19 10:44:25.774485 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:25.774450    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6" event={"ID":"60c115c6-2bfa-47eb-81ba-6620bb7239cc","Type":"ContainerStarted","Data":"6d8110ff84ac79650d5ac87fc55d04cdf5d12070902df9515d033ea4aa53246a"}
Sep 19 10:44:25.774852 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:25.774507    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6" event={"ID":"60c115c6-2bfa-47eb-81ba-6620bb7239cc","Type":"ContainerStarted","Data":"3c9877629fe2a92ee22204e8a150ad229f06f88e81b330ab9826c4de8316a4c4"}
Sep 19 10:44:25.793636 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:25.793608    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:44:25.798902 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:25.798862    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6" podStartSLOduration=1.627528699 podStartE2EDuration="1.798847239s" totalImagesPullingTime="171.31854ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:44:25.798082559 +0000 UTC m=+586.709776080" watchObservedRunningTime="2026-09-19 10:44:25.798847239 +0000 UTC m=+586.710540758"
Sep 19 10:44:30.913501 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:30.913468    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:44:32.606933 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:32.606904    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:44:34.564696 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:34.564660    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:34.565074 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:34.564725    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:34.577198 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:34.577169    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:34.830800 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:34.830718    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6"
Sep 19 10:44:39.649021 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.648986    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"]
Sep 19 10:44:39.655144 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.655122    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.658066 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.658045    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 19 10:44:39.663682 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.663652    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"]
Sep 19 10:44:39.692805 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.692783    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:44:39.845005 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.844981    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fa6796b-89df-460b-9275-0683e443326a-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.845133 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.845011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fa6796b-89df-460b-9275-0683e443326a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.845133 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.845051    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fa6796b-89df-460b-9275-0683e443326a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.845224 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.845142    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fa6796b-89df-460b-9275-0683e443326a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.845224 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.845216    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gksx8\" (UniqueName: \"kubernetes.io/projected/5fa6796b-89df-460b-9275-0683e443326a-kube-api-access-gksx8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.845309 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.845236    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fa6796b-89df-460b-9275-0683e443326a-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.946408 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.946378    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gksx8\" (UniqueName: \"kubernetes.io/projected/5fa6796b-89df-460b-9275-0683e443326a-kube-api-access-gksx8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.946583 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.946415    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fa6796b-89df-460b-9275-0683e443326a-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.946583 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.946439    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fa6796b-89df-460b-9275-0683e443326a-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.946583 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.946567    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fa6796b-89df-460b-9275-0683e443326a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.946751 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.946617    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fa6796b-89df-460b-9275-0683e443326a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.946751 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.946668    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fa6796b-89df-460b-9275-0683e443326a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.946871 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.946836    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fa6796b-89df-460b-9275-0683e443326a-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.946994 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.946967    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fa6796b-89df-460b-9275-0683e443326a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.947093 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.947033    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fa6796b-89df-460b-9275-0683e443326a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.948677 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.948659    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fa6796b-89df-460b-9275-0683e443326a-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.948960 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.948944    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fa6796b-89df-460b-9275-0683e443326a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.956496 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.956476    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gksx8\" (UniqueName: \"kubernetes.io/projected/5fa6796b-89df-460b-9275-0683e443326a-kube-api-access-gksx8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt\" (UID: \"5fa6796b-89df-460b-9275-0683e443326a\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:39.965317 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:39.965297    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:40.240159 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:40.240127    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"]
Sep 19 10:44:40.241571 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:44:40.241547    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5fa6796b_89df_460b_9275_0683e443326a.slice/crio-fdf8cfd0f736c27bfc8f20cb039f76e8533625871a76d0aaf615d3d2aa8e6457 WatchSource:0}: Error finding container fdf8cfd0f736c27bfc8f20cb039f76e8533625871a76d0aaf615d3d2aa8e6457: Status 404 returned error can't find the container with id fdf8cfd0f736c27bfc8f20cb039f76e8533625871a76d0aaf615d3d2aa8e6457
Sep 19 10:44:40.852968 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:40.852933    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt" event={"ID":"5fa6796b-89df-460b-9275-0683e443326a","Type":"ContainerStarted","Data":"1795694e5085480c2e2ae2c6796962889c865f309431e314ace2314fe6d540a5"}
Sep 19 10:44:40.852968 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:40.852973    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt" event={"ID":"5fa6796b-89df-460b-9275-0683e443326a","Type":"ContainerStarted","Data":"fdf8cfd0f736c27bfc8f20cb039f76e8533625871a76d0aaf615d3d2aa8e6457"}
Sep 19 10:44:40.870290 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:40.870242    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt" podStartSLOduration=1.709897488 podStartE2EDuration="1.870224007s" totalImagesPullingTime="160.326519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:44:40.870099671 +0000 UTC m=+601.781793195" watchObservedRunningTime="2026-09-19 10:44:40.870224007 +0000 UTC m=+601.781917526"
Sep 19 10:44:41.072673 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:41.072646    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:44:41.072841 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:41.072704    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:44:42.105378 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:42.105271    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:44:49.966053 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:49.966019    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:49.966544 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:49.966067    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:49.978544 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:49.978504    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:44:50.912657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:44:50.912627    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt"
Sep 19 10:45:00.165403 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:45:00.165325    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830245-n42n5"]
Sep 19 10:45:00.168719 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:45:00.168702    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830245-n42n5"
Sep 19 10:45:00.179853 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:45:00.179834    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mckfb\""
Sep 19 10:45:00.188421 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:45:00.188402    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830245-n42n5"
Sep 19 10:45:00.198952 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:45:00.198928    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830245-n42n5"]
Sep 19 10:45:00.461042 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:45:00.461019    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830245-n42n5"]
Sep 19 10:45:00.462732 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:45:00.462705    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71efe150_94ce_413c_98e1_29704b2587c7.slice/crio-70f925abab0dbc4fb7429596cdcd2c6f87646ddc0243434499e725560ba9fb00 WatchSource:0}: Error finding container 70f925abab0dbc4fb7429596cdcd2c6f87646ddc0243434499e725560ba9fb00: Status 404 returned error can't find the container with id 70f925abab0dbc4fb7429596cdcd2c6f87646ddc0243434499e725560ba9fb00
Sep 19 10:45:00.953673 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:45:00.953640    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830245-n42n5" event={"ID":"71efe150-94ce-413c-98e1-29704b2587c7","Type":"ContainerStarted","Data":"70f925abab0dbc4fb7429596cdcd2c6f87646ddc0243434499e725560ba9fb00"}
Sep 19 10:45:03.977197 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:45:03.977161    2579 generic.go:356] "Generic (PLEG): container finished" podID="71efe150-94ce-413c-98e1-29704b2587c7" containerID="7923589079d08f104580de7d77128953ef6987c85f703f5f097e9402fb43e998" exitCode=0
Sep 19 10:45:03.977580 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:45:03.977216    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830245-n42n5" event={"ID":"71efe150-94ce-413c-98e1-29704b2587c7","Type":"ContainerDied","Data":"7923589079d08f104580de7d77128953ef6987c85f703f5f097e9402fb43e998"}
Sep 19 10:45:05.109768 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:45:05.109737    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830245-n42n5"
Sep 19 10:45:05.988182 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:45:05.988151    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830245-n42n5"
Sep 19 10:45:05.988182 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:45:05.988158    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830245-n42n5" event={"ID":"71efe150-94ce-413c-98e1-29704b2587c7","Type":"ContainerDied","Data":"70f925abab0dbc4fb7429596cdcd2c6f87646ddc0243434499e725560ba9fb00"}
Sep 19 10:45:05.988570 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:45:05.988200    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70f925abab0dbc4fb7429596cdcd2c6f87646ddc0243434499e725560ba9fb00"
Sep 19 10:46:36.390562 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:46:36.390471    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:46:36.505963 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:46:36.505935    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:46:51.704783 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:46:51.704748    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:46:51.760243 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:46:51.760216    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:46:54.337478 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:46:54.337441    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:46:54.409506 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:46:54.409463    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:12.073815 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:12.073775    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:12.125834 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:12.125792    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:12.889150 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:12.889106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:13.220004 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:13.219973    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:14.321265 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:14.321236    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:14.416168 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:14.416132    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:15.411240 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:15.411208    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:15.510113 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:15.510080    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:16.433463 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:16.433432    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:29.766882 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:29.766846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:29.822421 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:29.822390    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:30.081815 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:30.081741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:30.154087 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:30.154051    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:30.220372 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:30.220334    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:30.370925 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:30.370857    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:30.478110 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:30.478082    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:30.623896 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:30.623816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:31.096823 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:31.096769    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:31.191856 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:31.191828    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:31.562344 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:31.562316    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:31.759902 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:31.759862    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9086-a3bc8f"]
Sep 19 10:47:31.760275 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:31.760258    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71efe150-94ce-413c-98e1-29704b2587c7" containerName="cleanup"
Sep 19 10:47:31.760357 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:31.760278    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="71efe150-94ce-413c-98e1-29704b2587c7" containerName="cleanup"
Sep 19 10:47:31.760409 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:31.760385    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="71efe150-94ce-413c-98e1-29704b2587c7" containerName="cleanup"
Sep 19 10:47:31.763239 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:31.763216    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9086-a3bc8f"
Sep 19 10:47:31.766010 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:31.765984    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9d6w5\""
Sep 19 10:47:31.767389 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:31.767361    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9086-a3bc8f"]
Sep 19 10:47:31.795153 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:31.795116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt892\" (UniqueName: \"kubernetes.io/projected/9205914c-f25d-4f90-b303-8d7619564a72-kube-api-access-mt892\") pod \"test-curl-9086-a3bc8f\" (UID: \"9205914c-f25d-4f90-b303-8d7619564a72\") " pod="opendatahub/test-curl-9086-a3bc8f"
Sep 19 10:47:31.895882 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:31.895797    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt892\" (UniqueName: \"kubernetes.io/projected/9205914c-f25d-4f90-b303-8d7619564a72-kube-api-access-mt892\") pod \"test-curl-9086-a3bc8f\" (UID: \"9205914c-f25d-4f90-b303-8d7619564a72\") " pod="opendatahub/test-curl-9086-a3bc8f"
Sep 19 10:47:31.903440 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:31.903415    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt892\" (UniqueName: \"kubernetes.io/projected/9205914c-f25d-4f90-b303-8d7619564a72-kube-api-access-mt892\") pod \"test-curl-9086-a3bc8f\" (UID: \"9205914c-f25d-4f90-b303-8d7619564a72\") " pod="opendatahub/test-curl-9086-a3bc8f"
Sep 19 10:47:32.074638 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:32.074608    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9086-a3bc8f"
Sep 19 10:47:32.341728 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:32.341702    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9086-a3bc8f"]
Sep 19 10:47:32.343945 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:47:32.343922    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9205914c_f25d_4f90_b303_8d7619564a72.slice/crio-8e4339ef974060988ff5f67793ddc13e327738985ab6f07a56d8938124feeba0 WatchSource:0}: Error finding container 8e4339ef974060988ff5f67793ddc13e327738985ab6f07a56d8938124feeba0: Status 404 returned error can't find the container with id 8e4339ef974060988ff5f67793ddc13e327738985ab6f07a56d8938124feeba0
Sep 19 10:47:32.345871 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:32.345850    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 10:47:32.733289 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:32.733257    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9086-a3bc8f" event={"ID":"9205914c-f25d-4f90-b303-8d7619564a72","Type":"ContainerStarted","Data":"8e4339ef974060988ff5f67793ddc13e327738985ab6f07a56d8938124feeba0"}
Sep 19 10:47:32.959360 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:32.959319    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:33.080896 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:33.080804    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:33.200576 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:33.200538    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:33.300331 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:33.300290    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:34.105375 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:34.105340    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:34.205799 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:34.205772    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:34.422171 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:34.422101    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:34.664333 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:34.664306    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:34.745171 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:34.745137    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9086-a3bc8f" event={"ID":"9205914c-f25d-4f90-b303-8d7619564a72","Type":"ContainerStarted","Data":"fd846a4df7e626ef5cb77ec19f9511b4f0705a180c1d8896897548bca37bbf35"}
Sep 19 10:47:34.758865 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:34.758825    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-9086-a3bc8f" podStartSLOduration=1.8667703420000001 podStartE2EDuration="3.758809636s" totalImagesPullingTime="1.892039294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:47:34.758034151 +0000 UTC m=+775.669727670" watchObservedRunningTime="2026-09-19 10:47:34.758809636 +0000 UTC m=+775.670503154"
Sep 19 10:47:34.763276 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:34.763255    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:35.147793 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.147761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:35.245882 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.245844    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:35.315307 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.315277    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9086-a3bc8f"]
Sep 19 10:47:35.318899 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.318877    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9086-a3bc8f"]
Sep 19 10:47:35.436370 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.436337    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9086-f6c128"]
Sep 19 10:47:35.436742 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.436725    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9205914c-f25d-4f90-b303-8d7619564a72" containerName="test-curl-9086-a3bc8f"
Sep 19 10:47:35.436834 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.436744    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9205914c-f25d-4f90-b303-8d7619564a72" containerName="test-curl-9086-a3bc8f"
Sep 19 10:47:35.436890 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.436843    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9205914c-f25d-4f90-b303-8d7619564a72" containerName="test-curl-9086-a3bc8f"
Sep 19 10:47:35.439921 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.439902    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9086-f6c128"
Sep 19 10:47:35.445622 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.445600    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9086-f6c128"]
Sep 19 10:47:35.522699 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.522672    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:35.528847 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.528828    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bq4k\" (UniqueName: \"kubernetes.io/projected/9927ac9c-93f9-42ef-bd4f-0998a832c2dd-kube-api-access-2bq4k\") pod \"test-curl-9086-f6c128\" (UID: \"9927ac9c-93f9-42ef-bd4f-0998a832c2dd\") " pod="opendatahub/test-curl-9086-f6c128"
Sep 19 10:47:35.629444 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.629416    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bq4k\" (UniqueName: \"kubernetes.io/projected/9927ac9c-93f9-42ef-bd4f-0998a832c2dd-kube-api-access-2bq4k\") pod \"test-curl-9086-f6c128\" (UID: \"9927ac9c-93f9-42ef-bd4f-0998a832c2dd\") " pod="opendatahub/test-curl-9086-f6c128"
Sep 19 10:47:35.637502 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.637477    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bq4k\" (UniqueName: \"kubernetes.io/projected/9927ac9c-93f9-42ef-bd4f-0998a832c2dd-kube-api-access-2bq4k\") pod \"test-curl-9086-f6c128\" (UID: \"9927ac9c-93f9-42ef-bd4f-0998a832c2dd\") " pod="opendatahub/test-curl-9086-f6c128"
Sep 19 10:47:35.688546 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.688468    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:35.750329 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.750298    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9086-f6c128"
Sep 19 10:47:35.750475 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:47:35.750410    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-9086-a3bc8f" container="test-curl-9086-a3bc8f"
Sep 19 10:47:35.787693 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:35.787655    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:36.019890 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.019865    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9086-f6c128"]
Sep 19 10:47:36.021561 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:47:36.021518    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9927ac9c_93f9_42ef_bd4f_0998a832c2dd.slice/crio-95d4a875befd8321916af2e6b097872bbf92deb2dd4242bb4ece0c656ddb80bb WatchSource:0}: Error finding container 95d4a875befd8321916af2e6b097872bbf92deb2dd4242bb4ece0c656ddb80bb: Status 404 returned error can't find the container with id 95d4a875befd8321916af2e6b097872bbf92deb2dd4242bb4ece0c656ddb80bb
Sep 19 10:47:36.161089 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.161057    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"]
Sep 19 10:47:36.164388 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.164372    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:36.167349 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.167329    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 10:47:36.167450 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.167366    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 10:47:36.168653 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.168632    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 10:47:36.168745 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.168632    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8b2hp\""
Sep 19 10:47:36.168811 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.168759    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-cc1b19-serving-cert\""
Sep 19 10:47:36.174587 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.174563    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"]
Sep 19 10:47:36.197082 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.197057    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:36.233368 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.233343    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-metrics-certs\") pod \"maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:36.233484 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.233371    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqmjd\" (UniqueName: \"kubernetes.io/projected/5a80d11d-2e3c-4a2c-886b-341c8debe269-kube-api-access-pqmjd\") pod \"maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:36.233484 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.233404    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a80d11d-2e3c-4a2c-886b-341c8debe269-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:36.233484 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.233455    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-maas-api-tls\") pod \"maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:36.257192 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.257170    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:36.334145 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.334062    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-metrics-certs\") pod \"maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:36.334145 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.334094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqmjd\" (UniqueName: \"kubernetes.io/projected/5a80d11d-2e3c-4a2c-886b-341c8debe269-kube-api-access-pqmjd\") pod \"maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:36.334145 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.334131    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a80d11d-2e3c-4a2c-886b-341c8debe269-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:36.334412 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.334155    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-maas-api-tls\") pod \"maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:36.334412 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:47:36.334254    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-serving-cert: secret "maas-api-e2e-worker-w0-cc1b19-serving-cert" not found
Sep 19 10:47:36.334412 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:47:36.334311    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-maas-api-tls podName:5a80d11d-2e3c-4a2c-886b-341c8debe269 nodeName:}" failed. No retries permitted until 2026-09-19 10:47:36.834293274 +0000 UTC m=+777.745986787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-maas-api-tls") pod "maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4" (UID: "5a80d11d-2e3c-4a2c-886b-341c8debe269") : secret "maas-api-e2e-worker-w0-cc1b19-serving-cert" not found
Sep 19 10:47:36.334907 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.334889    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a80d11d-2e3c-4a2c-886b-341c8debe269-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:36.336578 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.336557    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-metrics-certs\") pod \"maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:36.343488 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.343466    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqmjd\" (UniqueName: \"kubernetes.io/projected/5a80d11d-2e3c-4a2c-886b-341c8debe269-kube-api-access-pqmjd\") pod \"maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:36.458262 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.458237    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:36.668158 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.668087    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:36.738032 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.737986    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:36.757160 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.757128    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9086-f6c128" event={"ID":"9927ac9c-93f9-42ef-bd4f-0998a832c2dd","Type":"ContainerStarted","Data":"7bdee3dcd67e8120df58d815c4eca99552f7d71c3a4b3de52ac1f2f679bb9dc5"}
Sep 19 10:47:36.757295 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.757174    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9086-f6c128" event={"ID":"9927ac9c-93f9-42ef-bd4f-0998a832c2dd","Type":"ContainerStarted","Data":"95d4a875befd8321916af2e6b097872bbf92deb2dd4242bb4ece0c656ddb80bb"}
Sep 19 10:47:36.757338 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.757268    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-9086-a3bc8f" podUID="9205914c-f25d-4f90-b303-8d7619564a72" containerName="test-curl-9086-a3bc8f" containerID="cri-o://fd846a4df7e626ef5cb77ec19f9511b4f0705a180c1d8896897548bca37bbf35" gracePeriod=2
Sep 19 10:47:36.773214 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.773172    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-9086-f6c128" podStartSLOduration=1.773158058 podStartE2EDuration="1.773158058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:47:36.771419136 +0000 UTC m=+777.683112655" watchObservedRunningTime="2026-09-19 10:47:36.773158058 +0000 UTC m=+777.684851577"
Sep 19 10:47:36.773943 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:47:36.773920    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9086-a3bc8f\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="9205914c-f25d-4f90-b303-8d7619564a72" pod="opendatahub/test-curl-9086-a3bc8f"
Sep 19 10:47:36.838221 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.838193    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-maas-api-tls\") pod \"maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:36.840493 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:36.840471    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-maas-api-tls\") pod \"maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:37.076012 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:37.075969    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:37.173204 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:37.173164    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:37.350096 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:37.350025    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9086-f6c128"]
Sep 19 10:47:37.357440 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:37.357408    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"]
Sep 19 10:47:37.360240 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:37.360217    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9086-f6c128"]
Sep 19 10:47:37.762889 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:37.762852    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4" event={"ID":"5a80d11d-2e3c-4a2c-886b-341c8debe269","Type":"ContainerStarted","Data":"61dcd4d73b50b5ab2a15ab4d8a6a7b60f31f1e18af48ef55edae62fa86c1cd94"}
Sep 19 10:47:37.763044 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:47:37.763020    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-9086-f6c128" container="test-curl-9086-f6c128"
Sep 19 10:47:37.979452 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:37.979417    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:38.238675 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:38.238581    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:38.338978 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:38.338946    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:38.458431 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:38.458378    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:38.765464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:38.765435    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:38.768462 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:38.768430    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-9086-f6c128" podUID="9927ac9c-93f9-42ef-bd4f-0998a832c2dd" containerName="test-curl-9086-f6c128" containerID="cri-o://7bdee3dcd67e8120df58d815c4eca99552f7d71c3a4b3de52ac1f2f679bb9dc5" gracePeriod=2
Sep 19 10:47:38.770849 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:47:38.770823    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9086-f6c128\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="9927ac9c-93f9-42ef-bd4f-0998a832c2dd" pod="opendatahub/test-curl-9086-f6c128"
Sep 19 10:47:38.838231 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:38.838205    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:39.001968 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.001929    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:39.501333 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.501303    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:39.621586 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.621564    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9086-a3bc8f"
Sep 19 10:47:39.664017 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.663991    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt892\" (UniqueName: \"kubernetes.io/projected/9205914c-f25d-4f90-b303-8d7619564a72-kube-api-access-mt892\") pod \"9205914c-f25d-4f90-b303-8d7619564a72\" (UID: \"9205914c-f25d-4f90-b303-8d7619564a72\") "
Sep 19 10:47:39.666046 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.666026    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9205914c-f25d-4f90-b303-8d7619564a72-kube-api-access-mt892" (OuterVolumeSpecName: "kube-api-access-mt892") pod "9205914c-f25d-4f90-b303-8d7619564a72" (UID: "9205914c-f25d-4f90-b303-8d7619564a72"). InnerVolumeSpecName "kube-api-access-mt892". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:47:39.764569 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.764471    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt892\" (UniqueName: \"kubernetes.io/projected/9205914c-f25d-4f90-b303-8d7619564a72-kube-api-access-mt892\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:47:39.776229 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.776199    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4" event={"ID":"5a80d11d-2e3c-4a2c-886b-341c8debe269","Type":"ContainerStarted","Data":"68e60b1d1ba997f9bb4af2b909975a4abd1d173c07c7cda4f204a1ede022ca31"}
Sep 19 10:47:39.776395 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.776378    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:39.777514 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.777484    2579 generic.go:356] "Generic (PLEG): container finished" podID="9205914c-f25d-4f90-b303-8d7619564a72" containerID="fd846a4df7e626ef5cb77ec19f9511b4f0705a180c1d8896897548bca37bbf35" exitCode=137
Sep 19 10:47:39.777674 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.777513    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9086-a3bc8f"
Sep 19 10:47:39.777674 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.777572    2579 scope.go:117] "RemoveContainer" containerID="fd846a4df7e626ef5cb77ec19f9511b4f0705a180c1d8896897548bca37bbf35"
Sep 19 10:47:39.789751 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.789734    2579 scope.go:117] "RemoveContainer" containerID="fd846a4df7e626ef5cb77ec19f9511b4f0705a180c1d8896897548bca37bbf35"
Sep 19 10:47:39.790282 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:47:39.790260    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd846a4df7e626ef5cb77ec19f9511b4f0705a180c1d8896897548bca37bbf35\": container with ID starting with fd846a4df7e626ef5cb77ec19f9511b4f0705a180c1d8896897548bca37bbf35 not found: ID does not exist" containerID="fd846a4df7e626ef5cb77ec19f9511b4f0705a180c1d8896897548bca37bbf35"
Sep 19 10:47:39.790346 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.790294    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd846a4df7e626ef5cb77ec19f9511b4f0705a180c1d8896897548bca37bbf35"} err="failed to get container status \"fd846a4df7e626ef5cb77ec19f9511b4f0705a180c1d8896897548bca37bbf35\": rpc error: code = NotFound desc = could not find container \"fd846a4df7e626ef5cb77ec19f9511b4f0705a180c1d8896897548bca37bbf35\": container with ID starting with fd846a4df7e626ef5cb77ec19f9511b4f0705a180c1d8896897548bca37bbf35 not found: ID does not exist"
Sep 19 10:47:39.795535 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.795481    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4" podStartSLOduration=1.60796702 podStartE2EDuration="3.795467298s" totalImagesPullingTime="2.187500278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:47:39.794262359 +0000 UTC m=+780.705955879" watchObservedRunningTime="2026-09-19 10:47:39.795467298 +0000 UTC m=+780.707160816"
Sep 19 10:47:39.796471 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:47:39.796446    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9086-a3bc8f\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="9205914c-f25d-4f90-b303-8d7619564a72" pod="opendatahub/test-curl-9086-a3bc8f"
Sep 19 10:47:39.994213 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:39.994180    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:47:40.202720 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:40.202696    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:40.905127 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:40.905088    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:40.944880 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:40.944859    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9086-f6c128"
Sep 19 10:47:40.947513 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:47:40.947487    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9086-a3bc8f\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="9205914c-f25d-4f90-b303-8d7619564a72" pod="opendatahub/test-curl-9086-a3bc8f"
Sep 19 10:47:41.075235 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:41.075171    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bq4k\" (UniqueName: \"kubernetes.io/projected/9927ac9c-93f9-42ef-bd4f-0998a832c2dd-kube-api-access-2bq4k\") pod \"9927ac9c-93f9-42ef-bd4f-0998a832c2dd\" (UID: \"9927ac9c-93f9-42ef-bd4f-0998a832c2dd\") "
Sep 19 10:47:41.077158 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:41.077129    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9927ac9c-93f9-42ef-bd4f-0998a832c2dd-kube-api-access-2bq4k" (OuterVolumeSpecName: "kube-api-access-2bq4k") pod "9927ac9c-93f9-42ef-bd4f-0998a832c2dd" (UID: "9927ac9c-93f9-42ef-bd4f-0998a832c2dd"). InnerVolumeSpecName "kube-api-access-2bq4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:47:41.115002 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:47:41.114975    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9086-a3bc8f\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="9205914c-f25d-4f90-b303-8d7619564a72" pod="opendatahub/test-curl-9086-a3bc8f"
Sep 19 10:47:41.115658 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:41.115633    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9205914c-f25d-4f90-b303-8d7619564a72" path="/var/lib/kubelet/pods/9205914c-f25d-4f90-b303-8d7619564a72/volumes"
Sep 19 10:47:41.115956 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:41.115944    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9927ac9c-93f9-42ef-bd4f-0998a832c2dd" path="/var/lib/kubelet/pods/9927ac9c-93f9-42ef-bd4f-0998a832c2dd/volumes"
Sep 19 10:47:41.176256 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:41.176231    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bq4k\" (UniqueName: \"kubernetes.io/projected/9927ac9c-93f9-42ef-bd4f-0998a832c2dd-kube-api-access-2bq4k\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:47:41.365679 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:41.365608    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:41.471867 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:41.471839    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:41.701585 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:41.701544    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:41.790658 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:41.790624    2579 generic.go:356] "Generic (PLEG): container finished" podID="9927ac9c-93f9-42ef-bd4f-0998a832c2dd" containerID="7bdee3dcd67e8120df58d815c4eca99552f7d71c3a4b3de52ac1f2f679bb9dc5" exitCode=137
Sep 19 10:47:41.790850 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:41.790674    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9086-f6c128"
Sep 19 10:47:41.790850 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:41.790718    2579 scope.go:117] "RemoveContainer" containerID="7bdee3dcd67e8120df58d815c4eca99552f7d71c3a4b3de52ac1f2f679bb9dc5"
Sep 19 10:47:41.799076 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:41.799057    2579 scope.go:117] "RemoveContainer" containerID="7bdee3dcd67e8120df58d815c4eca99552f7d71c3a4b3de52ac1f2f679bb9dc5"
Sep 19 10:47:41.799347 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:47:41.799327    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bdee3dcd67e8120df58d815c4eca99552f7d71c3a4b3de52ac1f2f679bb9dc5\": container with ID starting with 7bdee3dcd67e8120df58d815c4eca99552f7d71c3a4b3de52ac1f2f679bb9dc5 not found: ID does not exist" containerID="7bdee3dcd67e8120df58d815c4eca99552f7d71c3a4b3de52ac1f2f679bb9dc5"
Sep 19 10:47:41.799391 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:41.799355    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bdee3dcd67e8120df58d815c4eca99552f7d71c3a4b3de52ac1f2f679bb9dc5"} err="failed to get container status \"7bdee3dcd67e8120df58d815c4eca99552f7d71c3a4b3de52ac1f2f679bb9dc5\": rpc error: code = NotFound desc = could not find container \"7bdee3dcd67e8120df58d815c4eca99552f7d71c3a4b3de52ac1f2f679bb9dc5\": container with ID starting with 7bdee3dcd67e8120df58d815c4eca99552f7d71c3a4b3de52ac1f2f679bb9dc5 not found: ID does not exist"
Sep 19 10:47:42.370281 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:42.370254    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:43.389315 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:43.389277    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:44.177479 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.177440    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"]
Sep 19 10:47:44.177786 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.177772    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9927ac9c-93f9-42ef-bd4f-0998a832c2dd" containerName="test-curl-9086-f6c128"
Sep 19 10:47:44.177786 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.177787    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9927ac9c-93f9-42ef-bd4f-0998a832c2dd" containerName="test-curl-9086-f6c128"
Sep 19 10:47:44.177911 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.177857    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9927ac9c-93f9-42ef-bd4f-0998a832c2dd" containerName="test-curl-9086-f6c128"
Sep 19 10:47:44.182440 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.182415    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:44.185241 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.185219    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b21bfa16-serving-cert\""
Sep 19 10:47:44.193231 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.193196    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"]
Sep 19 10:47:44.211291 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.211267    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:44.266540 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.266501    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:44.300741 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.300718    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e75e375-71e2-4a82-a343-0663e0ceab72-maas-api-tls\") pod \"maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:44.300839 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.300758    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e75e375-71e2-4a82-a343-0663e0ceab72-openshift-service-ca\") pod \"maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:44.300839 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.300792    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e75e375-71e2-4a82-a343-0663e0ceab72-metrics-certs\") pod \"maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:44.300921 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.300845    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fs98\" (UniqueName: \"kubernetes.io/projected/0e75e375-71e2-4a82-a343-0663e0ceab72-kube-api-access-4fs98\") pod \"maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:44.368160 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.368138    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:44.401515 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.401492    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e75e375-71e2-4a82-a343-0663e0ceab72-maas-api-tls\") pod \"maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:44.401864 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.401558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e75e375-71e2-4a82-a343-0663e0ceab72-openshift-service-ca\") pod \"maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:44.401864 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.401587    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e75e375-71e2-4a82-a343-0663e0ceab72-metrics-certs\") pod \"maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:44.401864 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.401621    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fs98\" (UniqueName: \"kubernetes.io/projected/0e75e375-71e2-4a82-a343-0663e0ceab72-kube-api-access-4fs98\") pod \"maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:44.402301 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.402284    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e75e375-71e2-4a82-a343-0663e0ceab72-openshift-service-ca\") pod \"maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:44.403770 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.403751    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e75e375-71e2-4a82-a343-0663e0ceab72-maas-api-tls\") pod \"maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:44.404005 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.403986    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e75e375-71e2-4a82-a343-0663e0ceab72-metrics-certs\") pod \"maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:44.409108 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.409084    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fs98\" (UniqueName: \"kubernetes.io/projected/0e75e375-71e2-4a82-a343-0663e0ceab72-kube-api-access-4fs98\") pod \"maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:44.494923 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.494854    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:44.698491 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.698463    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:44.765304 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.765279    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"]
Sep 19 10:47:44.767482 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:47:44.767453    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e75e375_71e2_4a82_a343_0663e0ceab72.slice/crio-f518b2910ecf982d9b46e90d2fbcf6c1bcf70df5a632b7c5925f55e70ee07ba8 WatchSource:0}: Error finding container f518b2910ecf982d9b46e90d2fbcf6c1bcf70df5a632b7c5925f55e70ee07ba8: Status 404 returned error can't find the container with id f518b2910ecf982d9b46e90d2fbcf6c1bcf70df5a632b7c5925f55e70ee07ba8
Sep 19 10:47:44.770181 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.770150    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:44.807999 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:44.807970    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl" event={"ID":"0e75e375-71e2-4a82-a343-0663e0ceab72","Type":"ContainerStarted","Data":"f518b2910ecf982d9b46e90d2fbcf6c1bcf70df5a632b7c5925f55e70ee07ba8"}
Sep 19 10:47:45.091760 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:45.091733    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:45.790263 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:45.790232    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:47:45.814770 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:45.814732    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl" event={"ID":"0e75e375-71e2-4a82-a343-0663e0ceab72","Type":"ContainerStarted","Data":"2ac10ae4883482f58a7021807c98fbb0dbce14dbe74caf1518b589ad57064fad"}
Sep 19 10:47:45.814931 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:45.814800    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:45.830821 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:45.830767    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl" podStartSLOduration=1.509184377 podStartE2EDuration="1.830748459s" totalImagesPullingTime="321.564082ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:47:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:47:45.829439771 +0000 UTC m=+786.741133291" watchObservedRunningTime="2026-09-19 10:47:45.830748459 +0000 UTC m=+786.742441979"
Sep 19 10:47:46.095007 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:46.094923    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:46.674268 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:46.674239    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:47.563957 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:47.563927    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:47.683736 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:47.683703    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:47.784045 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:47.784015    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:47.982732 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:47.982702    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:48.207903 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:48.207873    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:48.291614 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:48.291475    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:48.354195 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:48.354143    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:48.463698 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:48.463667    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:48.606334 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:48.606263    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:48.703209 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:48.703181    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:48.804327 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:48.804297    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:48.903173 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:48.903101    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:49.368996 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:49.368966    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:50.484958 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:50.484923    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:51.126187 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:51.126151    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:51.215065 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:51.215036    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:51.825687 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:51.825655    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:47:52.233123 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:52.233090    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:52.789284 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:52.789252    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:53.007174 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:53.007144    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:53.221562 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:53.221514    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:53.324365 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:53.324326    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:53.423760 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:53.423693    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:53.969290 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:53.969258    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"]
Sep 19 10:47:53.975707 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:53.975686    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:53.978294 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:53.978271    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-6cf6a0-serving-cert\""
Sep 19 10:47:53.985846 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:53.985822    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"]
Sep 19 10:47:54.013237 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.013211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:54.080678 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.080645    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef05874d-aa6b-411a-976f-6fa71e5328cf-openshift-service-ca\") pod \"maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:54.080841 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.080711    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef05874d-aa6b-411a-976f-6fa71e5328cf-maas-api-tls\") pod \"maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:54.080841 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.080733    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef05874d-aa6b-411a-976f-6fa71e5328cf-metrics-certs\") pod \"maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:54.080841 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.080759    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n8mj\" (UniqueName: \"kubernetes.io/projected/ef05874d-aa6b-411a-976f-6fa71e5328cf-kube-api-access-9n8mj\") pod \"maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:54.110459 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.110432    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:54.182048 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.182022    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n8mj\" (UniqueName: \"kubernetes.io/projected/ef05874d-aa6b-411a-976f-6fa71e5328cf-kube-api-access-9n8mj\") pod \"maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:54.182167 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.182068    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef05874d-aa6b-411a-976f-6fa71e5328cf-openshift-service-ca\") pod \"maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:54.182167 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.182114    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef05874d-aa6b-411a-976f-6fa71e5328cf-maas-api-tls\") pod \"maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:54.182167 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.182132    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef05874d-aa6b-411a-976f-6fa71e5328cf-metrics-certs\") pod \"maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:54.182808 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.182788    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef05874d-aa6b-411a-976f-6fa71e5328cf-openshift-service-ca\") pod \"maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:54.184487 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.184467    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef05874d-aa6b-411a-976f-6fa71e5328cf-maas-api-tls\") pod \"maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:54.184610 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.184595    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef05874d-aa6b-411a-976f-6fa71e5328cf-metrics-certs\") pod \"maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:54.189429 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.189410    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n8mj\" (UniqueName: \"kubernetes.io/projected/ef05874d-aa6b-411a-976f-6fa71e5328cf-kube-api-access-9n8mj\") pod \"maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:54.212234 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.212211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:54.288633 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.288561    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:54.503187 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.503160    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:54.563440 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.563413    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"]
Sep 19 10:47:54.565561 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.565517    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:54.566680 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:47:54.566647    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef05874d_aa6b_411a_976f_6fa71e5328cf.slice/crio-abfe3c842b35bee999b6c9a9cebed5f21f6704de6dfe5d9c992541c871c431a3 WatchSource:0}: Error finding container abfe3c842b35bee999b6c9a9cebed5f21f6704de6dfe5d9c992541c871c431a3: Status 404 returned error can't find the container with id abfe3c842b35bee999b6c9a9cebed5f21f6704de6dfe5d9c992541c871c431a3
Sep 19 10:47:54.861972 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.861889    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79" event={"ID":"ef05874d-aa6b-411a-976f-6fa71e5328cf","Type":"ContainerStarted","Data":"abfe3c842b35bee999b6c9a9cebed5f21f6704de6dfe5d9c992541c871c431a3"}
Sep 19 10:47:54.908232 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:54.908205    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:55.222272 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:55.222243    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:55.289345 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:55.289295    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:55.388478 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:55.388443    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:55.651313 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:55.651228    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:55.819255 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:55.819215    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:55.868317 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:55.868291    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79" event={"ID":"ef05874d-aa6b-411a-976f-6fa71e5328cf","Type":"ContainerStarted","Data":"10860d40feaae026a2dd489b0d99b9e5ff99712762988dabe0fc7fd23de7f30c"}
Sep 19 10:47:55.868448 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:55.868399    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:47:55.885968 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:55.885928    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79" podStartSLOduration=2.577097889 podStartE2EDuration="2.885914507s" totalImagesPullingTime="308.816618ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:47:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:47:55.885013301 +0000 UTC m=+796.796706821" watchObservedRunningTime="2026-09-19 10:47:55.885914507 +0000 UTC m=+796.797608026"
Sep 19 10:47:56.857868 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:56.857839    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:57.407260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:57.407228    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:58.175422 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:58.175391    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:58.936966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:58.936936    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:47:59.676484 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:47:59.676453    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:00.293446 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:00.293415    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:01.044157 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:01.044131    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:01.145177 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:01.145151    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:01.409672 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:01.409591    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:01.509368 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:01.509341    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:01.660964 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:01.660873    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:01.878839 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:01.878809    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:48:02.298044 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:02.298013    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:02.993952 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:02.993912    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:48:03.074794 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:03.074763    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:04.355930 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:04.355903    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:05.081871 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:05.081836    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:05.685268 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:05.685237    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:05.786419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:05.786387    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:06.387302 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:06.387272    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:06.994712 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:06.994675    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:07.640769 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:07.640735    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:08.231621 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.231589    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"]
Sep 19 10:48:08.233288 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.233249    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl" podUID="0e75e375-71e2-4a82-a343-0663e0ceab72" containerName="maas-api" containerID="cri-o://2ac10ae4883482f58a7021807c98fbb0dbce14dbe74caf1518b589ad57064fad" gracePeriod=30
Sep 19 10:48:08.265322 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.265297    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:08.364303 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.364276    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:08.435279 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.435248    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:08.479513 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.479491    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:48:08.487352 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.487270    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:08.582191 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.582157    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:08.600841 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.600792    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e75e375-71e2-4a82-a343-0663e0ceab72-maas-api-tls\") pod \"0e75e375-71e2-4a82-a343-0663e0ceab72\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") "
Sep 19 10:48:08.600979 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.600897    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e75e375-71e2-4a82-a343-0663e0ceab72-openshift-service-ca\") pod \"0e75e375-71e2-4a82-a343-0663e0ceab72\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") "
Sep 19 10:48:08.600979 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.600923    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e75e375-71e2-4a82-a343-0663e0ceab72-metrics-certs\") pod \"0e75e375-71e2-4a82-a343-0663e0ceab72\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") "
Sep 19 10:48:08.600979 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.600976    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fs98\" (UniqueName: \"kubernetes.io/projected/0e75e375-71e2-4a82-a343-0663e0ceab72-kube-api-access-4fs98\") pod \"0e75e375-71e2-4a82-a343-0663e0ceab72\" (UID: \"0e75e375-71e2-4a82-a343-0663e0ceab72\") "
Sep 19 10:48:08.601300 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.601280    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e75e375-71e2-4a82-a343-0663e0ceab72-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0e75e375-71e2-4a82-a343-0663e0ceab72" (UID: "0e75e375-71e2-4a82-a343-0663e0ceab72"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:48:08.603161 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.603134    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e75e375-71e2-4a82-a343-0663e0ceab72-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0e75e375-71e2-4a82-a343-0663e0ceab72" (UID: "0e75e375-71e2-4a82-a343-0663e0ceab72"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:48:08.603242 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.603172    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e75e375-71e2-4a82-a343-0663e0ceab72-kube-api-access-4fs98" (OuterVolumeSpecName: "kube-api-access-4fs98") pod "0e75e375-71e2-4a82-a343-0663e0ceab72" (UID: "0e75e375-71e2-4a82-a343-0663e0ceab72"). InnerVolumeSpecName "kube-api-access-4fs98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:48:08.603242 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.603179    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e75e375-71e2-4a82-a343-0663e0ceab72-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0e75e375-71e2-4a82-a343-0663e0ceab72" (UID: "0e75e375-71e2-4a82-a343-0663e0ceab72"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:48:08.701824 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.701793    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fs98\" (UniqueName: \"kubernetes.io/projected/0e75e375-71e2-4a82-a343-0663e0ceab72-kube-api-access-4fs98\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:48:08.701824 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.701822    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e75e375-71e2-4a82-a343-0663e0ceab72-maas-api-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:48:08.701824 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.701831    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e75e375-71e2-4a82-a343-0663e0ceab72-openshift-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:48:08.702038 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.701840    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e75e375-71e2-4a82-a343-0663e0ceab72-metrics-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:48:08.937030 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.936996    2579 generic.go:356] "Generic (PLEG): container finished" podID="0e75e375-71e2-4a82-a343-0663e0ceab72" containerID="2ac10ae4883482f58a7021807c98fbb0dbce14dbe74caf1518b589ad57064fad" exitCode=0
Sep 19 10:48:08.937186 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.937037    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"
Sep 19 10:48:08.937186 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.937083    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl" event={"ID":"0e75e375-71e2-4a82-a343-0663e0ceab72","Type":"ContainerDied","Data":"2ac10ae4883482f58a7021807c98fbb0dbce14dbe74caf1518b589ad57064fad"}
Sep 19 10:48:08.937186 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.937156    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl" event={"ID":"0e75e375-71e2-4a82-a343-0663e0ceab72","Type":"ContainerDied","Data":"f518b2910ecf982d9b46e90d2fbcf6c1bcf70df5a632b7c5925f55e70ee07ba8"}
Sep 19 10:48:08.937186 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.937179    2579 scope.go:117] "RemoveContainer" containerID="2ac10ae4883482f58a7021807c98fbb0dbce14dbe74caf1518b589ad57064fad"
Sep 19 10:48:08.946813 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.946798    2579 scope.go:117] "RemoveContainer" containerID="2ac10ae4883482f58a7021807c98fbb0dbce14dbe74caf1518b589ad57064fad"
Sep 19 10:48:08.947067 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:48:08.947042    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ac10ae4883482f58a7021807c98fbb0dbce14dbe74caf1518b589ad57064fad\": container with ID starting with 2ac10ae4883482f58a7021807c98fbb0dbce14dbe74caf1518b589ad57064fad not found: ID does not exist" containerID="2ac10ae4883482f58a7021807c98fbb0dbce14dbe74caf1518b589ad57064fad"
Sep 19 10:48:08.947130 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.947076    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ac10ae4883482f58a7021807c98fbb0dbce14dbe74caf1518b589ad57064fad"} err="failed to get container status \"2ac10ae4883482f58a7021807c98fbb0dbce14dbe74caf1518b589ad57064fad\": rpc error: code = NotFound desc = could not find container \"2ac10ae4883482f58a7021807c98fbb0dbce14dbe74caf1518b589ad57064fad\": container with ID starting with 2ac10ae4883482f58a7021807c98fbb0dbce14dbe74caf1518b589ad57064fad not found: ID does not exist"
Sep 19 10:48:08.958431 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.958411    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"]
Sep 19 10:48:08.963602 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:08.963581    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b21bfa16-8696dc57-5rxxl"]
Sep 19 10:48:09.035450 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:09.035424    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:09.114492 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:09.114462    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e75e375-71e2-4a82-a343-0663e0ceab72" path="/var/lib/kubelet/pods/0e75e375-71e2-4a82-a343-0663e0ceab72/volumes"
Sep 19 10:48:09.133009 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:09.132991    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:09.253507 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:09.253447    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:09.349690 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:09.349660    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:09.557311 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:09.557240    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:09.954251 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:09.954223    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:10.055500 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:10.055466    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:10.610881 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:10.610846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:11.393637 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:11.393604    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:11.705199 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:11.705168    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:11.801466 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:11.801438    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:12.154213 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.154140    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"]
Sep 19 10:48:12.154471 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.154459    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e75e375-71e2-4a82-a343-0663e0ceab72" containerName="maas-api"
Sep 19 10:48:12.154517 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.154472    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e75e375-71e2-4a82-a343-0663e0ceab72" containerName="maas-api"
Sep 19 10:48:12.154562 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.154552    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e75e375-71e2-4a82-a343-0663e0ceab72" containerName="maas-api"
Sep 19 10:48:12.157825 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.157808    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:12.160199 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.160174    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-fb988ee6-serving-cert\""
Sep 19 10:48:12.167257 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.167234    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"]
Sep 19 10:48:12.180922 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.180900    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:12.281118 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.281093    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:12.334193 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.334164    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-metrics-certs\") pod \"maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:12.334316 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.334202    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-maas-api-tls\") pod \"maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:12.334359 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.334312    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-openshift-service-ca\") pod \"maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:12.334359 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.334346    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wxd4\" (UniqueName: \"kubernetes.io/projected/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-kube-api-access-2wxd4\") pod \"maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:12.381596 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.381572    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:12.435761 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.435734    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-openshift-service-ca\") pod \"maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:12.435882 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.435763    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wxd4\" (UniqueName: \"kubernetes.io/projected/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-kube-api-access-2wxd4\") pod \"maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:12.435882 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.435808    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-metrics-certs\") pod \"maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:12.435882 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.435831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-maas-api-tls\") pod \"maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:12.436561 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.436515    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-openshift-service-ca\") pod \"maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:12.438257 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.438234    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-metrics-certs\") pod \"maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:12.438338 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.438287    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-maas-api-tls\") pod \"maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:12.443460 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.443439    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wxd4\" (UniqueName: \"kubernetes.io/projected/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-kube-api-access-2wxd4\") pod \"maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:12.469303 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.469276    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:12.642820 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.642780    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:12.746873 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.746849    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"]
Sep 19 10:48:12.748400 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:48:12.748366    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce4aced7_1ba8_4e9d_8bc0_c33e4b105915.slice/crio-5e35a18a7418780c28f8449e71bfc8eba91f71fa26eb5d75d57146ad6c747412 WatchSource:0}: Error finding container 5e35a18a7418780c28f8449e71bfc8eba91f71fa26eb5d75d57146ad6c747412: Status 404 returned error can't find the container with id 5e35a18a7418780c28f8449e71bfc8eba91f71fa26eb5d75d57146ad6c747412
Sep 19 10:48:12.748586 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.748468    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:12.960618 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:12.960585    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr" event={"ID":"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915","Type":"ContainerStarted","Data":"5e35a18a7418780c28f8449e71bfc8eba91f71fa26eb5d75d57146ad6c747412"}
Sep 19 10:48:13.003218 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:13.003142    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:13.672605 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:13.672570    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:13.967502 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:13.967410    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr" event={"ID":"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915","Type":"ContainerStarted","Data":"c1c3b58900bcfe17a93460a5a870dd52337f44d7ec9b8cd7d0e4769c9819ea0a"}
Sep 19 10:48:13.967502 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:13.967475    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:13.983665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:13.983612    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr" podStartSLOduration=1.543962287 podStartE2EDuration="1.983596303s" totalImagesPullingTime="439.634016ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:48:13.982468953 +0000 UTC m=+814.894162463" watchObservedRunningTime="2026-09-19 10:48:13.983596303 +0000 UTC m=+814.895289822"
Sep 19 10:48:14.199150 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:14.199093    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:14.993560 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:14.993509    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:15.595408 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:15.595365    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:16.323990 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.323957    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"]
Sep 19 10:48:16.327295 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.327278    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.331027 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.331002    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-cc1b19\"/\"kube-root-ca.crt\""
Sep 19 10:48:16.331027 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.331016    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-cc1b19\"/\"simulator-w0-cc1b19-kserve-self-signed-certs\""
Sep 19 10:48:16.331190 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.331009    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-cc1b19\"/\"openshift-service-ca.crt\""
Sep 19 10:48:16.331190 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.331012    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-cc1b19\"/\"default-dockercfg-865gf\""
Sep 19 10:48:16.336500 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.336481    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"]
Sep 19 10:48:16.367296 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.367267    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-tmp-dir\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.367403 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.367299    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-home\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.367403 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.367335    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-dshm\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.367403 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.367360    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckf2x\" (UniqueName: \"kubernetes.io/projected/6f7f478d-bb4a-42fc-96a2-33992bef3d04-kube-api-access-ckf2x\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.367602 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.367465    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7f478d-bb4a-42fc-96a2-33992bef3d04-tls-certs\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.367602 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.367561    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-model-cache\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.468364 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.468340    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-model-cache\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.468441 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.468369    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-tmp-dir\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.468441 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.468386    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-home\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.468441 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.468410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-dshm\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.468574 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.468558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckf2x\" (UniqueName: \"kubernetes.io/projected/6f7f478d-bb4a-42fc-96a2-33992bef3d04-kube-api-access-ckf2x\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.468671 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.468654    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7f478d-bb4a-42fc-96a2-33992bef3d04-tls-certs\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.468778 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.468755    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-model-cache\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.468922 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.468779    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-tmp-dir\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.469022 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.469002    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-home\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.470615 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.470592    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-dshm\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.470868 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.470852    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7f478d-bb4a-42fc-96a2-33992bef3d04-tls-certs\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.476330 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.476303    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckf2x\" (UniqueName: \"kubernetes.io/projected/6f7f478d-bb4a-42fc-96a2-33992bef3d04-kube-api-access-ckf2x\") pod \"simulator-w0-cc1b19-kserve-59857686d8-6jbmm\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") " pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.637927 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.637869    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:16.728160 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.728120    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:16.927059 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.927031    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"]
Sep 19 10:48:16.927562 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:48:16.927520    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f7f478d_bb4a_42fc_96a2_33992bef3d04.slice/crio-1f6bb7674a7fc270323e01a8d94116f6e1995f695d7ddc057d0d2a4f7336a3fc WatchSource:0}: Error finding container 1f6bb7674a7fc270323e01a8d94116f6e1995f695d7ddc057d0d2a4f7336a3fc: Status 404 returned error can't find the container with id 1f6bb7674a7fc270323e01a8d94116f6e1995f695d7ddc057d0d2a4f7336a3fc
Sep 19 10:48:16.983357 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:16.983326    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm" event={"ID":"6f7f478d-bb4a-42fc-96a2-33992bef3d04","Type":"ContainerStarted","Data":"1f6bb7674a7fc270323e01a8d94116f6e1995f695d7ddc057d0d2a4f7336a3fc"}
Sep 19 10:48:17.990034 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:17.989998    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm" event={"ID":"6f7f478d-bb4a-42fc-96a2-33992bef3d04","Type":"ContainerStarted","Data":"50668378b44408f7f35d888c2ed08ab6ccd3a1332b086dd19bad1db6100d7379"}
Sep 19 10:48:18.006777 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:18.006735    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm" podStartSLOduration=2.006720015 podStartE2EDuration="2.006720015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:48:18.004085262 +0000 UTC m=+818.915778780" watchObservedRunningTime="2026-09-19 10:48:18.006720015 +0000 UTC m=+818.918413535"
Sep 19 10:48:19.558952 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:19.558917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:19.977373 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:19.977343    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:23.398669 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:23.398636    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:48:26.638857 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:26.638815    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:26.638857 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:26.638863    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:26.651302 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:26.651273    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:27.025538 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.025490    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"]
Sep 19 10:48:27.029029 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.029010    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.031601 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.031577    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-580e01\"/\"simulator-w3-580e01-kserve-self-signed-certs\""
Sep 19 10:48:27.031601 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.031597    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-580e01\"/\"openshift-service-ca.crt\""
Sep 19 10:48:27.031784 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.031612    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-580e01\"/\"default-dockercfg-kml78\""
Sep 19 10:48:27.032722 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.032687    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-580e01\"/\"kube-root-ca.crt\""
Sep 19 10:48:27.036792 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.036299    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"]
Sep 19 10:48:27.052494 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.052474    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:48:27.056439 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.056421    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-home\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.056535 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.056458    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-model-cache\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.056587 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.056517    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-dshm\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.056622 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.056601    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9cea5d6-05e8-48ee-96ef-1808c82e953b-tls-certs\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.056656 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.056629    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhhpn\" (UniqueName: \"kubernetes.io/projected/e9cea5d6-05e8-48ee-96ef-1808c82e953b-kube-api-access-lhhpn\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.056691 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.056659    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-tmp-dir\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.157738 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.157710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-tmp-dir\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.157738 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.157744    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-home\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.157940 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.157797    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-model-cache\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.157940 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.157882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-dshm\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.158038 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.157964    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9cea5d6-05e8-48ee-96ef-1808c82e953b-tls-certs\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.158038 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.157991    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhhpn\" (UniqueName: \"kubernetes.io/projected/e9cea5d6-05e8-48ee-96ef-1808c82e953b-kube-api-access-lhhpn\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.158286 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.158255    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-tmp-dir\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.158395 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.158370    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-model-cache\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.158460 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.158394    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-home\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.160000 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.159980    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-dshm\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.160383 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.160363    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9cea5d6-05e8-48ee-96ef-1808c82e953b-tls-certs\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.165927 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.165903    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhhpn\" (UniqueName: \"kubernetes.io/projected/e9cea5d6-05e8-48ee-96ef-1808c82e953b-kube-api-access-lhhpn\") pod \"simulator-w3-580e01-kserve-57886c6ff-926nh\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") " pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.228430 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.228408    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:27.339678 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.339594    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:27.608871 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:27.608841    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"]
Sep 19 10:48:27.610697 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:48:27.610669    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9cea5d6_05e8_48ee_96ef_1808c82e953b.slice/crio-63cd91e033a3a75db668d63f0ef4281325ec3c79876191bd075aaa4f06fde60e WatchSource:0}: Error finding container 63cd91e033a3a75db668d63f0ef4281325ec3c79876191bd075aaa4f06fde60e: Status 404 returned error can't find the container with id 63cd91e033a3a75db668d63f0ef4281325ec3c79876191bd075aaa4f06fde60e
Sep 19 10:48:28.047990 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:28.047959    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh" event={"ID":"e9cea5d6-05e8-48ee-96ef-1808c82e953b","Type":"ContainerStarted","Data":"d869a26af09b06b80fb011ba15d4aed5617ac9942c0e9f890cbe0848b1e83067"}
Sep 19 10:48:28.048455 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:28.047998    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh" event={"ID":"e9cea5d6-05e8-48ee-96ef-1808c82e953b","Type":"ContainerStarted","Data":"63cd91e033a3a75db668d63f0ef4281325ec3c79876191bd075aaa4f06fde60e"}
Sep 19 10:48:28.063584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:28.063513    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh" podStartSLOduration=1.063498727 podStartE2EDuration="1.063498727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:48:28.062636484 +0000 UTC m=+828.974329995" watchObservedRunningTime="2026-09-19 10:48:28.063498727 +0000 UTC m=+828.975192270"
Sep 19 10:48:36.262471 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:36.262419    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:37.339699 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:37.339661    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:37.340102 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:37.339712    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:37.352492 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:37.352473    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:37.461914 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:37.461878    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:38.117925 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:38.117899    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:48:38.177771 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:38.177744    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:38.772321 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:38.772291    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:39.364260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:39.364231    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:40.104815 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:40.104782    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:40.716364 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:40.716337    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:41.240122 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.240091    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"]
Sep 19 10:48:41.245083 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.245066    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.248885 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.248863    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-f91f97\"/\"test-model-f91f97-kserve-self-signed-certs\""
Sep 19 10:48:41.248885 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.248875    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-f91f97\"/\"kube-root-ca.crt\""
Sep 19 10:48:41.249015 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.248876    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-f91f97\"/\"openshift-service-ca.crt\""
Sep 19 10:48:41.249015 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.248869    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-f91f97\"/\"default-dockercfg-6h46n\""
Sep 19 10:48:41.254419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.254396    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"]
Sep 19 10:48:41.262991 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.262970    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:41.385959 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.385898    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-tmp-dir\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.385959 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.385940    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-dshm\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.386091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.385963    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-model-cache\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.386091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.386057    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-home\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.386156 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.386089    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b47f44-d649-44c6-ae56-e92a67c29516-tls-certs\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.386156 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.386122    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq2kn\" (UniqueName: \"kubernetes.io/projected/a3b47f44-d649-44c6-ae56-e92a67c29516-kube-api-access-fq2kn\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.487302 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.487275    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq2kn\" (UniqueName: \"kubernetes.io/projected/a3b47f44-d649-44c6-ae56-e92a67c29516-kube-api-access-fq2kn\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.487406 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.487306    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-tmp-dir\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.487406 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.487330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-dshm\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.487479 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.487413    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-model-cache\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.487518 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.487504    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-home\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.487588 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.487554    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b47f44-d649-44c6-ae56-e92a67c29516-tls-certs\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.487686 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.487671    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-tmp-dir\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.487855 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.487837    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-home\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.487915 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.487898    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-model-cache\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.489570 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.489543    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-dshm\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.489844 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.489829    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b47f44-d649-44c6-ae56-e92a67c29516-tls-certs\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.495550 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.495511    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq2kn\" (UniqueName: \"kubernetes.io/projected/a3b47f44-d649-44c6-ae56-e92a67c29516-kube-api-access-fq2kn\") pod \"test-model-f91f97-kserve-6859df89c7-z6q5k\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") " pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.554657 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.554635    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:41.823028 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:41.822999    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"]
Sep 19 10:48:41.824473 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:48:41.824447    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3b47f44_d649_44c6_ae56_e92a67c29516.slice/crio-4dd37ac0a57cd924ec9c8deaca63556f9fd7951b6277f80a33b97491c87e6580 WatchSource:0}: Error finding container 4dd37ac0a57cd924ec9c8deaca63556f9fd7951b6277f80a33b97491c87e6580: Status 404 returned error can't find the container with id 4dd37ac0a57cd924ec9c8deaca63556f9fd7951b6277f80a33b97491c87e6580
Sep 19 10:48:42.135197 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:42.135105    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k" event={"ID":"a3b47f44-d649-44c6-ae56-e92a67c29516","Type":"ContainerStarted","Data":"44999f4bcbbc1aa035b57dd187644ae9891ea3de3dee573017926217843d4486"}
Sep 19 10:48:42.135197 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:42.135153    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k" event={"ID":"a3b47f44-d649-44c6-ae56-e92a67c29516","Type":"ContainerStarted","Data":"4dd37ac0a57cd924ec9c8deaca63556f9fd7951b6277f80a33b97491c87e6580"}
Sep 19 10:48:42.149974 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:42.149924    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k" podStartSLOduration=1.149910723 podStartE2EDuration="1.149910723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:48:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:48:42.149174412 +0000 UTC m=+843.060867931" watchObservedRunningTime="2026-09-19 10:48:42.149910723 +0000 UTC m=+843.061604242"
Sep 19 10:48:42.566462 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:42.566432    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:43.203444 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:43.203412    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:43.939593 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:43.939557    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:44.465641 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:44.465615    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:45.053163 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:45.053130    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:45.711794 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:45.711764    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:48:45.767208 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:45.767179    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:51.555123 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:51.555088    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:51.555508 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:51.555139    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:51.567385 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:51.567364    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:52.197628 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:52.197598    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:48:53.213633 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.213600    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"]
Sep 19 10:48:53.214023 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.213857    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr" podUID="ce4aced7-1ba8-4e9d-8bc0-c33e4b105915" containerName="maas-api" containerID="cri-o://c1c3b58900bcfe17a93460a5a870dd52337f44d7ec9b8cd7d0e4769c9819ea0a" gracePeriod=30
Sep 19 10:48:53.238225 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.238204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:53.339326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.339284    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:53.419115 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.419087    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:53.452670 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.452651    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:53.476433 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.476379    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:53.552875 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.552851    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:53.597296 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.597270    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-metrics-certs\") pod \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") "
Sep 19 10:48:53.597425 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.597309    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wxd4\" (UniqueName: \"kubernetes.io/projected/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-kube-api-access-2wxd4\") pod \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") "
Sep 19 10:48:53.597425 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.597379    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-maas-api-tls\") pod \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") "
Sep 19 10:48:53.597507 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.597425    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-openshift-service-ca\") pod \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\" (UID: \"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915\") "
Sep 19 10:48:53.597869 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.597832    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ce4aced7-1ba8-4e9d-8bc0-c33e4b105915" (UID: "ce4aced7-1ba8-4e9d-8bc0-c33e4b105915"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:48:53.599409 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.599380    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ce4aced7-1ba8-4e9d-8bc0-c33e4b105915" (UID: "ce4aced7-1ba8-4e9d-8bc0-c33e4b105915"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:48:53.599499 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.599443    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-kube-api-access-2wxd4" (OuterVolumeSpecName: "kube-api-access-2wxd4") pod "ce4aced7-1ba8-4e9d-8bc0-c33e4b105915" (UID: "ce4aced7-1ba8-4e9d-8bc0-c33e4b105915"). InnerVolumeSpecName "kube-api-access-2wxd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:48:53.599499 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.599464    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ce4aced7-1ba8-4e9d-8bc0-c33e4b105915" (UID: "ce4aced7-1ba8-4e9d-8bc0-c33e4b105915"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:48:53.698187 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.698159    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-maas-api-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:48:53.698187 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.698184    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-openshift-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:48:53.698340 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.698194    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-metrics-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:48:53.698340 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.698204    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wxd4\" (UniqueName: \"kubernetes.io/projected/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915-kube-api-access-2wxd4\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:48:53.844382 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.844306    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:53.989158 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:53.989130    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:54.198738 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:54.198701    2579 generic.go:356] "Generic (PLEG): container finished" podID="ce4aced7-1ba8-4e9d-8bc0-c33e4b105915" containerID="c1c3b58900bcfe17a93460a5a870dd52337f44d7ec9b8cd7d0e4769c9819ea0a" exitCode=0
Sep 19 10:48:54.198907 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:54.198747    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"
Sep 19 10:48:54.198907 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:54.198785    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr" event={"ID":"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915","Type":"ContainerDied","Data":"c1c3b58900bcfe17a93460a5a870dd52337f44d7ec9b8cd7d0e4769c9819ea0a"}
Sep 19 10:48:54.198907 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:54.198840    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr" event={"ID":"ce4aced7-1ba8-4e9d-8bc0-c33e4b105915","Type":"ContainerDied","Data":"5e35a18a7418780c28f8449e71bfc8eba91f71fa26eb5d75d57146ad6c747412"}
Sep 19 10:48:54.198907 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:54.198865    2579 scope.go:117] "RemoveContainer" containerID="c1c3b58900bcfe17a93460a5a870dd52337f44d7ec9b8cd7d0e4769c9819ea0a"
Sep 19 10:48:54.209889 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:54.209873    2579 scope.go:117] "RemoveContainer" containerID="c1c3b58900bcfe17a93460a5a870dd52337f44d7ec9b8cd7d0e4769c9819ea0a"
Sep 19 10:48:54.210316 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:48:54.210296    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1c3b58900bcfe17a93460a5a870dd52337f44d7ec9b8cd7d0e4769c9819ea0a\": container with ID starting with c1c3b58900bcfe17a93460a5a870dd52337f44d7ec9b8cd7d0e4769c9819ea0a not found: ID does not exist" containerID="c1c3b58900bcfe17a93460a5a870dd52337f44d7ec9b8cd7d0e4769c9819ea0a"
Sep 19 10:48:54.210387 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:54.210328    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1c3b58900bcfe17a93460a5a870dd52337f44d7ec9b8cd7d0e4769c9819ea0a"} err="failed to get container status \"c1c3b58900bcfe17a93460a5a870dd52337f44d7ec9b8cd7d0e4769c9819ea0a\": rpc error: code = NotFound desc = could not find container \"c1c3b58900bcfe17a93460a5a870dd52337f44d7ec9b8cd7d0e4769c9819ea0a\": container with ID starting with c1c3b58900bcfe17a93460a5a870dd52337f44d7ec9b8cd7d0e4769c9819ea0a not found: ID does not exist"
Sep 19 10:48:54.224495 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:54.224476    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"]
Sep 19 10:48:54.231885 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:54.231865    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fb988ee6-596f7886f6-tfsdr"]
Sep 19 10:48:54.286755 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:54.286727    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:54.503984 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:54.503900    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:55.116210 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:55.116177    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce4aced7-1ba8-4e9d-8bc0-c33e4b105915" path="/var/lib/kubelet/pods/ce4aced7-1ba8-4e9d-8bc0-c33e4b105915/volumes"
Sep 19 10:48:55.136075 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:55.136021    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:55.248649 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:55.248623    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:55.980429 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:55.980384    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:56.670380 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:56.670346    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:56.988474 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:56.988445    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:57.086487 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:57.086460    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:57.398401 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:57.398315    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:58.274028 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:58.274000    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:58.940260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:58.940235    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:48:59.663715 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:48:59.663685    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:00.152735 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.152698    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"]
Sep 19 10:49:00.153296 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.153280    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce4aced7-1ba8-4e9d-8bc0-c33e4b105915" containerName="maas-api"
Sep 19 10:49:00.153378 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.153299    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce4aced7-1ba8-4e9d-8bc0-c33e4b105915" containerName="maas-api"
Sep 19 10:49:00.153414 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.153400    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce4aced7-1ba8-4e9d-8bc0-c33e4b105915" containerName="maas-api"
Sep 19 10:49:00.156854 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.156832    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:00.159317 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.159294    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-963aea34-serving-cert\""
Sep 19 10:49:00.166228 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.166203    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"]
Sep 19 10:49:00.181274 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.181235    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:00.253132 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.253108    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f42f059a-1987-412c-8a4e-ccd01cb7c48d-metrics-certs\") pod \"maas-api-e2e-ait-963aea34-74b66d5848-frg7v\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:00.253255 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.253165    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f42f059a-1987-412c-8a4e-ccd01cb7c48d-maas-api-tls\") pod \"maas-api-e2e-ait-963aea34-74b66d5848-frg7v\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:00.253255 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.253190    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f42f059a-1987-412c-8a4e-ccd01cb7c48d-openshift-service-ca\") pod \"maas-api-e2e-ait-963aea34-74b66d5848-frg7v\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:00.253255 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.253207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t849\" (UniqueName: \"kubernetes.io/projected/f42f059a-1987-412c-8a4e-ccd01cb7c48d-kube-api-access-8t849\") pod \"maas-api-e2e-ait-963aea34-74b66d5848-frg7v\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:00.281643 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.281585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:00.354221 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.354194    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f42f059a-1987-412c-8a4e-ccd01cb7c48d-metrics-certs\") pod \"maas-api-e2e-ait-963aea34-74b66d5848-frg7v\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:00.354319 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.354255    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f42f059a-1987-412c-8a4e-ccd01cb7c48d-maas-api-tls\") pod \"maas-api-e2e-ait-963aea34-74b66d5848-frg7v\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:00.354319 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.354283    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f42f059a-1987-412c-8a4e-ccd01cb7c48d-openshift-service-ca\") pod \"maas-api-e2e-ait-963aea34-74b66d5848-frg7v\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:00.354319 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.354309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t849\" (UniqueName: \"kubernetes.io/projected/f42f059a-1987-412c-8a4e-ccd01cb7c48d-kube-api-access-8t849\") pod \"maas-api-e2e-ait-963aea34-74b66d5848-frg7v\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:00.354982 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.354957    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f42f059a-1987-412c-8a4e-ccd01cb7c48d-openshift-service-ca\") pod \"maas-api-e2e-ait-963aea34-74b66d5848-frg7v\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:00.356788 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.356769    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f42f059a-1987-412c-8a4e-ccd01cb7c48d-maas-api-tls\") pod \"maas-api-e2e-ait-963aea34-74b66d5848-frg7v\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:00.356872 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.356773    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f42f059a-1987-412c-8a4e-ccd01cb7c48d-metrics-certs\") pod \"maas-api-e2e-ait-963aea34-74b66d5848-frg7v\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:00.361314 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.361295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t849\" (UniqueName: \"kubernetes.io/projected/f42f059a-1987-412c-8a4e-ccd01cb7c48d-kube-api-access-8t849\") pod \"maas-api-e2e-ait-963aea34-74b66d5848-frg7v\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:00.383812 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.383785    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:00.468832 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.468805    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:00.619743 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.619713    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:00.709930 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.709894    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:00.740726 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:00.740701    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"]
Sep 19 10:49:00.742338 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:49:00.742317    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf42f059a_1987_412c_8a4e_ccd01cb7c48d.slice/crio-f0bd0f22829faa50cb56b33874e9474a91e5ced270b216af4dc8002c1dc80d9f WatchSource:0}: Error finding container f0bd0f22829faa50cb56b33874e9474a91e5ced270b216af4dc8002c1dc80d9f: Status 404 returned error can't find the container with id f0bd0f22829faa50cb56b33874e9474a91e5ced270b216af4dc8002c1dc80d9f
Sep 19 10:49:01.240700 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:01.240666    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v" event={"ID":"f42f059a-1987-412c-8a4e-ccd01cb7c48d","Type":"ContainerStarted","Data":"18ec39aedf7c2529d6346aa1ad9a9f2489fee2e1a2a7072de280d39ab41848a9"}
Sep 19 10:49:01.240843 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:01.240714    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v" event={"ID":"f42f059a-1987-412c-8a4e-ccd01cb7c48d","Type":"ContainerStarted","Data":"f0bd0f22829faa50cb56b33874e9474a91e5ced270b216af4dc8002c1dc80d9f"}
Sep 19 10:49:01.240843 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:01.240747    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:01.257336 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:01.257291    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v" podStartSLOduration=0.878476106 podStartE2EDuration="1.257275796s" totalImagesPullingTime="378.79969ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:49:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:49:01.256160996 +0000 UTC m=+862.167854515" watchObservedRunningTime="2026-09-19 10:49:01.257275796 +0000 UTC m=+862.168969314"
Sep 19 10:49:07.252456 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:07.252432    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:07.793405 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:07.793370    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:49:22.570725 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:22.570690    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:23.180213 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:23.180186    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:23.864351 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:23.864321    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:24.369093 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:24.369066    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:24.985400 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:24.985372    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:25.778390 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:25.778351    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:28.373839 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:28.373807    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:28.845541 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:28.845482    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"]
Sep 19 10:49:28.848705 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:28.848686    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:28.851184 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:28.851160    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-cc1b19\"/\"premium-w0-cc1b19-kserve-self-signed-certs\""
Sep 19 10:49:28.856617 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:28.856591    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"]
Sep 19 10:49:28.900842 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:28.900809    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-model-cache\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:28.900842 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:28.900844    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-tmp-dir\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:28.901056 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:28.900862    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f44ebb5c-d9ba-4898-947f-d1507a3986ff-tls-certs\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:28.901056 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:28.900915    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-home\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:28.901056 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:28.900952    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnrl4\" (UniqueName: \"kubernetes.io/projected/f44ebb5c-d9ba-4898-947f-d1507a3986ff-kube-api-access-xnrl4\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:28.901056 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:28.901036    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-dshm\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:28.946026 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:28.945997    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:29.001798 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.001767    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-model-cache\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:29.001943 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.001802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-tmp-dir\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:29.001943 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.001822    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f44ebb5c-d9ba-4898-947f-d1507a3986ff-tls-certs\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:29.001943 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.001869    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-home\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:29.001943 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.001892    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnrl4\" (UniqueName: \"kubernetes.io/projected/f44ebb5c-d9ba-4898-947f-d1507a3986ff-kube-api-access-xnrl4\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:29.002154 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.001953    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-dshm\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:29.002239 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.002218    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-model-cache\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:29.002299 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.002251    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-tmp-dir\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:29.002299 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.002289    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-home\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:29.004077 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.004058    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-dshm\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:29.004364 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.004345    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f44ebb5c-d9ba-4898-947f-d1507a3986ff-tls-certs\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:29.010009 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.009986    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnrl4\" (UniqueName: \"kubernetes.io/projected/f44ebb5c-d9ba-4898-947f-d1507a3986ff-kube-api-access-xnrl4\") pod \"premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") " pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:29.070925 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.070900    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:29.158767 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.158695    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:29.428393 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.428368    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"]
Sep 19 10:49:29.430315 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:49:29.430291    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf44ebb5c_d9ba_4898_947f_d1507a3986ff.slice/crio-d0e4dc7a329eeddea6f511004a87d313fa5311391391e1fe1c8afc7bb800fb12 WatchSource:0}: Error finding container d0e4dc7a329eeddea6f511004a87d313fa5311391391e1fe1c8afc7bb800fb12: Status 404 returned error can't find the container with id d0e4dc7a329eeddea6f511004a87d313fa5311391391e1fe1c8afc7bb800fb12
Sep 19 10:49:29.686323 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.686241    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:29.950074 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:29.949986    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:30.049935 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:30.049908    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:30.371486 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:30.371403    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:30.397065 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:30.397040    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4" event={"ID":"f44ebb5c-d9ba-4898-947f-d1507a3986ff","Type":"ContainerStarted","Data":"aaf31506e802760d9b30e4de999af482fda76a0ced85ac55d4a6331340c977e8"}
Sep 19 10:49:30.397211 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:30.397079    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4" event={"ID":"f44ebb5c-d9ba-4898-947f-d1507a3986ff","Type":"ContainerStarted","Data":"d0e4dc7a329eeddea6f511004a87d313fa5311391391e1fe1c8afc7bb800fb12"}
Sep 19 10:49:30.415826 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:30.415775    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4" podStartSLOduration=2.415761727 podStartE2EDuration="2.415761727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:49:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:49:30.413251487 +0000 UTC m=+891.324944998" watchObservedRunningTime="2026-09-19 10:49:30.415761727 +0000 UTC m=+891.327455245"
Sep 19 10:49:30.968079 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:30.968039    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:31.541337 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:31.541299    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:34.595495 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:34.595461    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:49:35.338382 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:35.338348    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:38.541862 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.541828    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"]
Sep 19 10:49:38.542273 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.542173    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v" podUID="f42f059a-1987-412c-8a4e-ccd01cb7c48d" containerName="maas-api" containerID="cri-o://18ec39aedf7c2529d6346aa1ad9a9f2489fee2e1a2a7072de280d39ab41848a9" gracePeriod=30
Sep 19 10:49:38.563865 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.563832    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:38.626048 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.626016    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:38.729348 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.729309    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:38.785672 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.785650    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:38.814298 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.814199    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:38.882399 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.882374    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8t849\" (UniqueName: \"kubernetes.io/projected/f42f059a-1987-412c-8a4e-ccd01cb7c48d-kube-api-access-8t849\") pod \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") "
Sep 19 10:49:38.882568 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.882447    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f42f059a-1987-412c-8a4e-ccd01cb7c48d-openshift-service-ca\") pod \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") "
Sep 19 10:49:38.882568 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.882508    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f42f059a-1987-412c-8a4e-ccd01cb7c48d-metrics-certs\") pod \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") "
Sep 19 10:49:38.882652 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.882577    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f42f059a-1987-412c-8a4e-ccd01cb7c48d-maas-api-tls\") pod \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\" (UID: \"f42f059a-1987-412c-8a4e-ccd01cb7c48d\") "
Sep 19 10:49:38.882859 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.882835    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f42f059a-1987-412c-8a4e-ccd01cb7c48d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f42f059a-1987-412c-8a4e-ccd01cb7c48d" (UID: "f42f059a-1987-412c-8a4e-ccd01cb7c48d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:49:38.884648 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.884621    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f42f059a-1987-412c-8a4e-ccd01cb7c48d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f42f059a-1987-412c-8a4e-ccd01cb7c48d" (UID: "f42f059a-1987-412c-8a4e-ccd01cb7c48d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:49:38.884742 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.884695    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f42f059a-1987-412c-8a4e-ccd01cb7c48d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f42f059a-1987-412c-8a4e-ccd01cb7c48d" (UID: "f42f059a-1987-412c-8a4e-ccd01cb7c48d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:49:38.884742 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.884711    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f42f059a-1987-412c-8a4e-ccd01cb7c48d-kube-api-access-8t849" (OuterVolumeSpecName: "kube-api-access-8t849") pod "f42f059a-1987-412c-8a4e-ccd01cb7c48d" (UID: "f42f059a-1987-412c-8a4e-ccd01cb7c48d"). InnerVolumeSpecName "kube-api-access-8t849". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:49:38.983765 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.983741    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f42f059a-1987-412c-8a4e-ccd01cb7c48d-openshift-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:49:38.983765 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.983765    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f42f059a-1987-412c-8a4e-ccd01cb7c48d-metrics-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:49:38.983909 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.983774    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f42f059a-1987-412c-8a4e-ccd01cb7c48d-maas-api-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:49:38.983909 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:38.983782    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8t849\" (UniqueName: \"kubernetes.io/projected/f42f059a-1987-412c-8a4e-ccd01cb7c48d-kube-api-access-8t849\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:49:39.087294 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.087216    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:39.158889 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.158852    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:39.159032 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.158899    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:39.171091 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.171072    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:39.446291 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.446259    2579 generic.go:356] "Generic (PLEG): container finished" podID="f42f059a-1987-412c-8a4e-ccd01cb7c48d" containerID="18ec39aedf7c2529d6346aa1ad9a9f2489fee2e1a2a7072de280d39ab41848a9" exitCode=0
Sep 19 10:49:39.446473 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.446308    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"
Sep 19 10:49:39.446473 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.446337    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v" event={"ID":"f42f059a-1987-412c-8a4e-ccd01cb7c48d","Type":"ContainerDied","Data":"18ec39aedf7c2529d6346aa1ad9a9f2489fee2e1a2a7072de280d39ab41848a9"}
Sep 19 10:49:39.446473 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.446385    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v" event={"ID":"f42f059a-1987-412c-8a4e-ccd01cb7c48d","Type":"ContainerDied","Data":"f0bd0f22829faa50cb56b33874e9474a91e5ced270b216af4dc8002c1dc80d9f"}
Sep 19 10:49:39.446473 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.446405    2579 scope.go:117] "RemoveContainer" containerID="18ec39aedf7c2529d6346aa1ad9a9f2489fee2e1a2a7072de280d39ab41848a9"
Sep 19 10:49:39.455578 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.455557    2579 scope.go:117] "RemoveContainer" containerID="18ec39aedf7c2529d6346aa1ad9a9f2489fee2e1a2a7072de280d39ab41848a9"
Sep 19 10:49:39.455864 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:49:39.455841    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18ec39aedf7c2529d6346aa1ad9a9f2489fee2e1a2a7072de280d39ab41848a9\": container with ID starting with 18ec39aedf7c2529d6346aa1ad9a9f2489fee2e1a2a7072de280d39ab41848a9 not found: ID does not exist" containerID="18ec39aedf7c2529d6346aa1ad9a9f2489fee2e1a2a7072de280d39ab41848a9"
Sep 19 10:49:39.455926 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.455874    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ec39aedf7c2529d6346aa1ad9a9f2489fee2e1a2a7072de280d39ab41848a9"} err="failed to get container status \"18ec39aedf7c2529d6346aa1ad9a9f2489fee2e1a2a7072de280d39ab41848a9\": rpc error: code = NotFound desc = could not find container \"18ec39aedf7c2529d6346aa1ad9a9f2489fee2e1a2a7072de280d39ab41848a9\": container with ID starting with 18ec39aedf7c2529d6346aa1ad9a9f2489fee2e1a2a7072de280d39ab41848a9 not found: ID does not exist"
Sep 19 10:49:39.458268 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.458249    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:49:39.462613 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.462592    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"]
Sep 19 10:49:39.469227 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.469207    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-963aea34-74b66d5848-frg7v"]
Sep 19 10:49:39.627455 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.627420    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:39.839558 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:39.839467    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:40.261894 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:40.261866    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:40.375630 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:40.375602    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:40.560459 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:40.560381    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:40.869192 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:40.869106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:41.114893 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:41.114864    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:49:41.116179 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:41.116144    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f42f059a-1987-412c-8a4e-ccd01cb7c48d" path="/var/lib/kubelet/pods/f42f059a-1987-412c-8a4e-ccd01cb7c48d/volumes"
Sep 19 10:49:41.117197 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:41.117179    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:49:41.931224 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:41.931195    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"]
Sep 19 10:49:41.931586 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:41.931566    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f42f059a-1987-412c-8a4e-ccd01cb7c48d" containerName="maas-api"
Sep 19 10:49:41.931586 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:41.931578    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f42f059a-1987-412c-8a4e-ccd01cb7c48d" containerName="maas-api"
Sep 19 10:49:41.931671 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:41.931637    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f42f059a-1987-412c-8a4e-ccd01cb7c48d" containerName="maas-api"
Sep 19 10:49:41.935957 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:41.935934    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:41.939908 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:41.939886    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-6cf6a0\"/\"openshift-service-ca.crt\""
Sep 19 10:49:41.940049 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:41.939891    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-6cf6a0\"/\"kube-root-ca.crt\""
Sep 19 10:49:41.940049 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:41.939910    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-6cf6a0\"/\"default-dockercfg-zkppl\""
Sep 19 10:49:41.940049 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:41.939968    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-6cf6a0\"/\"test-model-6cf6a0-kserve-self-signed-certs\""
Sep 19 10:49:41.945946 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:41.945923    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"]
Sep 19 10:49:42.011115 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.011082    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2cf1c1a-2156-470b-9248-fcfa21685809-tls-certs\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.011115 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.011113    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlmkq\" (UniqueName: \"kubernetes.io/projected/a2cf1c1a-2156-470b-9248-fcfa21685809-kube-api-access-mlmkq\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.011320 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.011139    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-dshm\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.011320 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.011168    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-tmp-dir\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.011320 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.011266    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-home\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.011320 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.011316    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-model-cache\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.112171 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.112061    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2cf1c1a-2156-470b-9248-fcfa21685809-tls-certs\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.112171 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.112105    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlmkq\" (UniqueName: \"kubernetes.io/projected/a2cf1c1a-2156-470b-9248-fcfa21685809-kube-api-access-mlmkq\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.112171 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.112138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-dshm\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.112171 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.112169    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-tmp-dir\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.129345 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.112210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-home\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.129345 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.112263    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-model-cache\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.129345 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.112601    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-tmp-dir\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.129345 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.112620    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-model-cache\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.129345 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.112651    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-home\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.129345 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.114410    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-dshm\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.129345 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.114564    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2cf1c1a-2156-470b-9248-fcfa21685809-tls-certs\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.129345 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.119393    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlmkq\" (UniqueName: \"kubernetes.io/projected/a2cf1c1a-2156-470b-9248-fcfa21685809-kube-api-access-mlmkq\") pod \"test-model-6cf6a0-kserve-59cbd7c765-s4phf\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") " pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.208627 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.208543    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:42.246665 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.246639    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:42.515087 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.515064    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"]
Sep 19 10:49:42.516268 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:49:42.516242    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2cf1c1a_2156_470b_9248_fcfa21685809.slice/crio-7e9007200c9910fb9e42e1ad1609586159904c40a1fff1a74350d3e84ef94c74 WatchSource:0}: Error finding container 7e9007200c9910fb9e42e1ad1609586159904c40a1fff1a74350d3e84ef94c74: Status 404 returned error can't find the container with id 7e9007200c9910fb9e42e1ad1609586159904c40a1fff1a74350d3e84ef94c74
Sep 19 10:49:42.568230 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:42.568206    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:43.086012 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:43.085981    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:43.239006 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:43.238972    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:43.338975 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:43.338904    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:43.475028 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:43.474987    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf" event={"ID":"a2cf1c1a-2156-470b-9248-fcfa21685809","Type":"ContainerStarted","Data":"6b8e3849efec88c091a74edb2fc4d22c1e41e9f702cebe6da013a221432953fc"}
Sep 19 10:49:43.475028 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:43.475027    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf" event={"ID":"a2cf1c1a-2156-470b-9248-fcfa21685809","Type":"ContainerStarted","Data":"7e9007200c9910fb9e42e1ad1609586159904c40a1fff1a74350d3e84ef94c74"}
Sep 19 10:49:43.491616 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:43.491570    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf" podStartSLOduration=2.491552308 podStartE2EDuration="2.491552308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:49:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:49:43.490295212 +0000 UTC m=+904.401988722" watchObservedRunningTime="2026-09-19 10:49:43.491552308 +0000 UTC m=+904.403245826"
Sep 19 10:49:43.965025 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:43.964992    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:44.600123 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:44.600093    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:45.269808 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:45.269777    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:45.876586 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:45.876548    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"]
Sep 19 10:49:45.880702 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:45.880680    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:45.883141 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:45.883117    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-219b8581-serving-cert\""
Sep 19 10:49:45.889912 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:45.889881    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"]
Sep 19 10:49:45.903461 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:45.903426    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:46.004495 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.004465    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:46.049169 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.049144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b32d859e-055e-4a8a-a930-fbd9386d2ab4-maas-api-tls\") pod \"maas-api-e2e-derive-219b8581-788fcc578c-s8l6c\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:46.049293 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.049199    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4t6n\" (UniqueName: \"kubernetes.io/projected/b32d859e-055e-4a8a-a930-fbd9386d2ab4-kube-api-access-x4t6n\") pod \"maas-api-e2e-derive-219b8581-788fcc578c-s8l6c\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:46.049293 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.049279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b32d859e-055e-4a8a-a930-fbd9386d2ab4-openshift-service-ca\") pod \"maas-api-e2e-derive-219b8581-788fcc578c-s8l6c\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:46.049363 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.049310    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b32d859e-055e-4a8a-a930-fbd9386d2ab4-metrics-certs\") pod \"maas-api-e2e-derive-219b8581-788fcc578c-s8l6c\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:46.122964 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.122939    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:46.150260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.150196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4t6n\" (UniqueName: \"kubernetes.io/projected/b32d859e-055e-4a8a-a930-fbd9386d2ab4-kube-api-access-x4t6n\") pod \"maas-api-e2e-derive-219b8581-788fcc578c-s8l6c\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:46.150260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.150242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b32d859e-055e-4a8a-a930-fbd9386d2ab4-openshift-service-ca\") pod \"maas-api-e2e-derive-219b8581-788fcc578c-s8l6c\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:46.150260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.150259    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b32d859e-055e-4a8a-a930-fbd9386d2ab4-metrics-certs\") pod \"maas-api-e2e-derive-219b8581-788fcc578c-s8l6c\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:46.150481 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.150434    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b32d859e-055e-4a8a-a930-fbd9386d2ab4-maas-api-tls\") pod \"maas-api-e2e-derive-219b8581-788fcc578c-s8l6c\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:46.150912 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.150886    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b32d859e-055e-4a8a-a930-fbd9386d2ab4-openshift-service-ca\") pod \"maas-api-e2e-derive-219b8581-788fcc578c-s8l6c\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:46.152674 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.152653    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b32d859e-055e-4a8a-a930-fbd9386d2ab4-metrics-certs\") pod \"maas-api-e2e-derive-219b8581-788fcc578c-s8l6c\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:46.152765 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.152707    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b32d859e-055e-4a8a-a930-fbd9386d2ab4-maas-api-tls\") pod \"maas-api-e2e-derive-219b8581-788fcc578c-s8l6c\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:46.158271 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.158253    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4t6n\" (UniqueName: \"kubernetes.io/projected/b32d859e-055e-4a8a-a930-fbd9386d2ab4-kube-api-access-x4t6n\") pod \"maas-api-e2e-derive-219b8581-788fcc578c-s8l6c\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:46.194560 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.194518    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:46.410438 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.410358    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:46.465230 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.465203    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"]
Sep 19 10:49:46.466982 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:49:46.466954    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb32d859e_055e_4a8a_a930_fbd9386d2ab4.slice/crio-d01b283a12dce8c97e30a91d19327e780a6927882dfc80890887df871c49c4a4 WatchSource:0}: Error finding container d01b283a12dce8c97e30a91d19327e780a6927882dfc80890887df871c49c4a4: Status 404 returned error can't find the container with id d01b283a12dce8c97e30a91d19327e780a6927882dfc80890887df871c49c4a4
Sep 19 10:49:46.468291 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.468266    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:49:46.491143 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:46.491119    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c" event={"ID":"b32d859e-055e-4a8a-a930-fbd9386d2ab4","Type":"ContainerStarted","Data":"d01b283a12dce8c97e30a91d19327e780a6927882dfc80890887df871c49c4a4"}
Sep 19 10:49:47.497745 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:47.497707    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c" event={"ID":"b32d859e-055e-4a8a-a930-fbd9386d2ab4","Type":"ContainerStarted","Data":"8002de15909b8f6745ea9e1ae261fc25984f6db5386796a12cd8934271818217"}
Sep 19 10:49:47.498147 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:47.497825    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:49:47.515159 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:47.515113    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c" podStartSLOduration=2.16788479 podStartE2EDuration="2.515099496s" totalImagesPullingTime="347.214706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:49:47.512982908 +0000 UTC m=+908.424676441" watchObservedRunningTime="2026-09-19 10:49:47.515099496 +0000 UTC m=+908.426793014"
Sep 19 10:49:52.247140 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:52.247110    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:52.247538 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:52.247266    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:52.259737 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:52.259711    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:52.536237 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:52.536162    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:49:53.509286 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:49:53.509258    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:50:03.301737 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:03.301698    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:50:08.142778 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:08.142749    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:08.771316 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:08.771287    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:09.490976 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:09.490944    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:10.272931 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:10.272894    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:10.889384 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:10.889355    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:11.602403 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:11.602372    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:12.257262 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:12.257232    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:14.559153 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:14.559122    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:15.167814 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.167784    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"]
Sep 19 10:50:15.168009 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.167988    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c" podUID="b32d859e-055e-4a8a-a930-fbd9386d2ab4" containerName="maas-api" containerID="cri-o://8002de15909b8f6745ea9e1ae261fc25984f6db5386796a12cd8934271818217" gracePeriod=30
Sep 19 10:50:15.195712 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.195669    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:15.254778 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.254740    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:15.314560 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.314517    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:15.370855 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.370824    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:15.413019 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.412999    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:50:15.427452 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.427428    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:15.508351 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.508321    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:15.521430 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.521409    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4t6n\" (UniqueName: \"kubernetes.io/projected/b32d859e-055e-4a8a-a930-fbd9386d2ab4-kube-api-access-x4t6n\") pod \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") "
Sep 19 10:50:15.521553 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.521489    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b32d859e-055e-4a8a-a930-fbd9386d2ab4-metrics-certs\") pod \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") "
Sep 19 10:50:15.521599 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.521549    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b32d859e-055e-4a8a-a930-fbd9386d2ab4-maas-api-tls\") pod \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") "
Sep 19 10:50:15.521599 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.521574    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b32d859e-055e-4a8a-a930-fbd9386d2ab4-openshift-service-ca\") pod \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\" (UID: \"b32d859e-055e-4a8a-a930-fbd9386d2ab4\") "
Sep 19 10:50:15.521916 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.521888    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b32d859e-055e-4a8a-a930-fbd9386d2ab4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b32d859e-055e-4a8a-a930-fbd9386d2ab4" (UID: "b32d859e-055e-4a8a-a930-fbd9386d2ab4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:50:15.523575 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.523518    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b32d859e-055e-4a8a-a930-fbd9386d2ab4-kube-api-access-x4t6n" (OuterVolumeSpecName: "kube-api-access-x4t6n") pod "b32d859e-055e-4a8a-a930-fbd9386d2ab4" (UID: "b32d859e-055e-4a8a-a930-fbd9386d2ab4"). InnerVolumeSpecName "kube-api-access-x4t6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:50:15.523575 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.523551    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b32d859e-055e-4a8a-a930-fbd9386d2ab4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b32d859e-055e-4a8a-a930-fbd9386d2ab4" (UID: "b32d859e-055e-4a8a-a930-fbd9386d2ab4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:50:15.523705 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.523632    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b32d859e-055e-4a8a-a930-fbd9386d2ab4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b32d859e-055e-4a8a-a930-fbd9386d2ab4" (UID: "b32d859e-055e-4a8a-a930-fbd9386d2ab4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:50:15.622641 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.622604    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b32d859e-055e-4a8a-a930-fbd9386d2ab4-metrics-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:50:15.622641 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.622635    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b32d859e-055e-4a8a-a930-fbd9386d2ab4-maas-api-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:50:15.622641 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.622646    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b32d859e-055e-4a8a-a930-fbd9386d2ab4-openshift-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:50:15.623044 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.622656    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4t6n\" (UniqueName: \"kubernetes.io/projected/b32d859e-055e-4a8a-a930-fbd9386d2ab4-kube-api-access-x4t6n\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:50:15.649325 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.649294    2579 generic.go:356] "Generic (PLEG): container finished" podID="b32d859e-055e-4a8a-a930-fbd9386d2ab4" containerID="8002de15909b8f6745ea9e1ae261fc25984f6db5386796a12cd8934271818217" exitCode=0
Sep 19 10:50:15.649482 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.649345    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"
Sep 19 10:50:15.649482 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.649380    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c" event={"ID":"b32d859e-055e-4a8a-a930-fbd9386d2ab4","Type":"ContainerDied","Data":"8002de15909b8f6745ea9e1ae261fc25984f6db5386796a12cd8934271818217"}
Sep 19 10:50:15.649482 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.649431    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c" event={"ID":"b32d859e-055e-4a8a-a930-fbd9386d2ab4","Type":"ContainerDied","Data":"d01b283a12dce8c97e30a91d19327e780a6927882dfc80890887df871c49c4a4"}
Sep 19 10:50:15.649482 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.649455    2579 scope.go:117] "RemoveContainer" containerID="8002de15909b8f6745ea9e1ae261fc25984f6db5386796a12cd8934271818217"
Sep 19 10:50:15.666051 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.666030    2579 scope.go:117] "RemoveContainer" containerID="8002de15909b8f6745ea9e1ae261fc25984f6db5386796a12cd8934271818217"
Sep 19 10:50:15.666319 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:50:15.666300    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8002de15909b8f6745ea9e1ae261fc25984f6db5386796a12cd8934271818217\": container with ID starting with 8002de15909b8f6745ea9e1ae261fc25984f6db5386796a12cd8934271818217 not found: ID does not exist" containerID="8002de15909b8f6745ea9e1ae261fc25984f6db5386796a12cd8934271818217"
Sep 19 10:50:15.666370 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.666328    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8002de15909b8f6745ea9e1ae261fc25984f6db5386796a12cd8934271818217"} err="failed to get container status \"8002de15909b8f6745ea9e1ae261fc25984f6db5386796a12cd8934271818217\": rpc error: code = NotFound desc = could not find container \"8002de15909b8f6745ea9e1ae261fc25984f6db5386796a12cd8934271818217\": container with ID starting with 8002de15909b8f6745ea9e1ae261fc25984f6db5386796a12cd8934271818217 not found: ID does not exist"
Sep 19 10:50:15.675140 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.675113    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"]
Sep 19 10:50:15.680808 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.680760    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-219b8581-788fcc578c-s8l6c"]
Sep 19 10:50:15.682062 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:15.682046    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:16.163403 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:16.163317    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:16.231828 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:16.231803    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:16.341385 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:16.341357    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:16.583111 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:16.583082    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:17.116162 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:17.116133    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b32d859e-055e-4a8a-a930-fbd9386d2ab4" path="/var/lib/kubelet/pods/b32d859e-055e-4a8a-a930-fbd9386d2ab4/volumes"
Sep 19 10:50:17.352782 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:17.352748    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:17.693070 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:17.693042    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:18.058674 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:18.058599    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:18.663066 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:18.663034    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:19.698469 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:19.698442    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:23.440589 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:23.440550    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:23.498783 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:23.498758    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:23.575070 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:23.575044    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:23.673896 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:23.673871    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:23.775067 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:23.775006    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:24.102231 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:24.102157    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:35.501458 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:35.501422    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:50:50.005159 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:50.005103    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:55.447179 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.447102    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"]
Sep 19 10:50:55.447683 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.447664    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b32d859e-055e-4a8a-a930-fbd9386d2ab4" containerName="maas-api"
Sep 19 10:50:55.447783 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.447686    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b32d859e-055e-4a8a-a930-fbd9386d2ab4" containerName="maas-api"
Sep 19 10:50:55.447869 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.447857    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b32d859e-055e-4a8a-a930-fbd9386d2ab4" containerName="maas-api"
Sep 19 10:50:55.452388 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.452368    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.455152 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.455131    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-cc1b19\"/\"distinct-w0-cc1b19-kserve-self-signed-certs\""
Sep 19 10:50:55.460552 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.460513    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"]
Sep 19 10:50:55.465448 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.465427    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:55.563282 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.563255    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-home\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.563420 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.563289    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-tmp-dir\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.563420 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.563393    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-model-cache\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.563520 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.563433    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-dshm\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.563520 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.563451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-tls-certs\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.563520 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.563482    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbztr\" (UniqueName: \"kubernetes.io/projected/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-kube-api-access-tbztr\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.664244 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.664193    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbztr\" (UniqueName: \"kubernetes.io/projected/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-kube-api-access-tbztr\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.664429 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.664257    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-home\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.664429 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.664279    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-tmp-dir\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.664429 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.664372    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-model-cache\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.664429 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.664415    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-dshm\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.664706 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.664437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-tls-certs\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.664767 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.664712    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-home\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.664767 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.664737    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-tmp-dir\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.664870 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.664834    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-model-cache\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.666760 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.666738    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-dshm\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.666889 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.666873    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-tls-certs\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.671582 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.671563    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbztr\" (UniqueName: \"kubernetes.io/projected/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-kube-api-access-tbztr\") pod \"distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.760551 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.760460    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:55.762608 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.762590    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:50:55.816350 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.816314    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:55.887511 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.887485    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:55.990313 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:55.990279    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:50:56.037640 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:56.037615    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"]
Sep 19 10:50:56.038568 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:50:56.038539    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd9a7db1_d0be_4d01_ae33_fe2f087fcad3.slice/crio-d989c31413acdbf78f753bacfa7980f081502a13673d7426f38b77aa2fad8596 WatchSource:0}: Error finding container d989c31413acdbf78f753bacfa7980f081502a13673d7426f38b77aa2fad8596: Status 404 returned error can't find the container with id d989c31413acdbf78f753bacfa7980f081502a13673d7426f38b77aa2fad8596
Sep 19 10:50:56.878519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:56.878484    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m" event={"ID":"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3","Type":"ContainerStarted","Data":"d66b9aefc7a53e5f3492c8f85468723d88e849abf4d676fbbb109ba3736d3c97"}
Sep 19 10:50:56.878519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:56.878548    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m" event={"ID":"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3","Type":"ContainerStarted","Data":"d989c31413acdbf78f753bacfa7980f081502a13673d7426f38b77aa2fad8596"}
Sep 19 10:50:56.897561 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:50:56.897500    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m" podStartSLOduration=1.897485612 podStartE2EDuration="1.897485612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:50:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:50:56.895996652 +0000 UTC m=+977.807690177" watchObservedRunningTime="2026-09-19 10:50:56.897485612 +0000 UTC m=+977.809179133"
Sep 19 10:51:01.303165 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:01.303128    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:51:05.763170 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:05.763135    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:51:05.763560 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:05.763180    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:51:05.775499 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:05.775479    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:51:05.935376 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:05.935352    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:51:11.752850 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:11.752809    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:12.783447 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:12.783415    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:17.452659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:17.452623    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:17.506648 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:17.506617    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:17.573688 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:17.573659    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:17.690036 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:17.690005    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:24.403396 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:24.403354    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:51:28.138100 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:28.138037    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:28.889473 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:28.889442    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:33.532684 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.532652    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"]
Sep 19 10:51:33.536265 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.536246    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.539137 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.539115    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-cc1b19\"/\"distinct-2-w0-cc1b19-kserve-self-signed-certs\""
Sep 19 10:51:33.545851 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.545829    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"]
Sep 19 10:51:33.557360 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.557337    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:33.696346 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.696318    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-tmp-dir\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.696482 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.696363    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x64dx\" (UniqueName: \"kubernetes.io/projected/39c264fc-e8e7-4e35-979d-a0c035e67696-kube-api-access-x64dx\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.696544 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.696502    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-home\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.696589 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.696560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-model-cache\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.696589 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.696580    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-dshm\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.696655 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.696606    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39c264fc-e8e7-4e35-979d-a0c035e67696-tls-certs\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.797230 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.797145    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x64dx\" (UniqueName: \"kubernetes.io/projected/39c264fc-e8e7-4e35-979d-a0c035e67696-kube-api-access-x64dx\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.797375 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.797254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-home\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.797375 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.797287    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-model-cache\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.797375 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.797312    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-dshm\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.797375 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.797349    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39c264fc-e8e7-4e35-979d-a0c035e67696-tls-certs\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.797593 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.797380    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-tmp-dir\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.797709 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.797690    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-model-cache\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.797803 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.797785    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-home\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.797864 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.797842    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-tmp-dir\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.799652 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.799630    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-dshm\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.799849 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.799831    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39c264fc-e8e7-4e35-979d-a0c035e67696-tls-certs\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.804535 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.804497    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x64dx\" (UniqueName: \"kubernetes.io/projected/39c264fc-e8e7-4e35-979d-a0c035e67696-kube-api-access-x64dx\") pod \"distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") " pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:33.846331 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:33.846305    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:34.115736 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:34.115701    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"]
Sep 19 10:51:34.117399 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:51:34.117370    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39c264fc_e8e7_4e35_979d_a0c035e67696.slice/crio-44f28bdecda5f33fbde98f6eed3414d7ef2d773344261d2038a8cd7865e8ea6c WatchSource:0}: Error finding container 44f28bdecda5f33fbde98f6eed3414d7ef2d773344261d2038a8cd7865e8ea6c: Status 404 returned error can't find the container with id 44f28bdecda5f33fbde98f6eed3414d7ef2d773344261d2038a8cd7865e8ea6c
Sep 19 10:51:34.744751 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:34.744719    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:34.798296 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:34.798256    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:34.874925 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:34.874884    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:34.973862 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:34.973831    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:35.074579 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:35.074505    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:35.078075 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:35.078050    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h" event={"ID":"39c264fc-e8e7-4e35-979d-a0c035e67696","Type":"ContainerStarted","Data":"c250180a6b935bb5c346ea401ceeb88b3b1671085fb62a264307a5100defadb9"}
Sep 19 10:51:35.078251 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:35.078084    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h" event={"ID":"39c264fc-e8e7-4e35-979d-a0c035e67696","Type":"ContainerStarted","Data":"44f28bdecda5f33fbde98f6eed3414d7ef2d773344261d2038a8cd7865e8ea6c"}
Sep 19 10:51:35.093871 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:35.093831    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h" podStartSLOduration=2.093817702 podStartE2EDuration="2.093817702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:51:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:51:35.092939643 +0000 UTC m=+1016.004633156" watchObservedRunningTime="2026-09-19 10:51:35.093817702 +0000 UTC m=+1016.005511269"
Sep 19 10:51:35.665790 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:35.665757    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:35.757825 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:35.757795    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:35.857983 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:35.857951    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:37.284169 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:37.284138    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:42.995276 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:42.995247    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:43.053519 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:43.053486    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:43.639623 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:43.639587    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:43.847325 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:43.847297    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:43.847487 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:43.847363    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:43.860554 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:43.860514    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:44.140791 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:44.140765    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:51:44.205877 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:44.205837    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:51:46.069545 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:46.069500    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:46.123323 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:46.123293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:49.555804 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:49.555765    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:49.655901 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:49.655872    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:49.790932 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:49.790906    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:50.089964 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:50.089925    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:50.231551 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:50.231493    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:52.770018 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:52.769986    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:53.368814 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:53.368777    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:53.519216 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:53.519181    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:53.619230 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:53.619152    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:54.108819 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:54.108776    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:54.872016 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:54.871981    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:55.542158 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:55.542125    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:56.271062 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:56.271029    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:56.323410 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:56.323368    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:56.807252 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:56.807198    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:56.907672 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:56.907647    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:57.158164 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:57.158091    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:51:57.675121 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:51:57.675090    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:11.637490 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.637451    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"]
Sep 19 10:52:11.641213 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.641187    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.643783 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.643762    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-580e01\"/\"distinct-2-w3-580e01-kserve-self-signed-certs\""
Sep 19 10:52:11.648240 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.648216    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"]
Sep 19 10:52:11.657991 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.657954    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:11.745845 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.745817    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tptxw\" (UniqueName: \"kubernetes.io/projected/70e95713-3c73-468c-ae4d-bf785a758838-kube-api-access-tptxw\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.745981 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.745860    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-dshm\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.746048 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.745980    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-home\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.746106 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.746061    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70e95713-3c73-468c-ae4d-bf785a758838-tls-certs\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.746106 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.746096    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-tmp-dir\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.746193 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.746140    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-model-cache\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.847130 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.847099    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tptxw\" (UniqueName: \"kubernetes.io/projected/70e95713-3c73-468c-ae4d-bf785a758838-kube-api-access-tptxw\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.847260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.847141    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-dshm\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.847260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.847171    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-home\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.847260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.847223    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70e95713-3c73-468c-ae4d-bf785a758838-tls-certs\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.847260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.847253    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-tmp-dir\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.847441 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.847388    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-model-cache\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.847673 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.847648    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-tmp-dir\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.847788 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.847694    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-home\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.847843 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.847784    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-model-cache\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.849551 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.849511    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-dshm\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.849797 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.849781    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70e95713-3c73-468c-ae4d-bf785a758838-tls-certs\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.855236 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.855214    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tptxw\" (UniqueName: \"kubernetes.io/projected/70e95713-3c73-468c-ae4d-bf785a758838-kube-api-access-tptxw\") pod \"distinct-2-w3-580e01-kserve-f99856f9d-hz8kn\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") " pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:11.951859 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:11.951833    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:12.221679 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:12.221651    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"]
Sep 19 10:52:12.223256 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:52:12.223223    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70e95713_3c73_468c_ae4d_bf785a758838.slice/crio-fd0f5243dece3cdfb9cec1db67602c2cff2238ac73101e07d158fd81f0cdd319 WatchSource:0}: Error finding container fd0f5243dece3cdfb9cec1db67602c2cff2238ac73101e07d158fd81f0cdd319: Status 404 returned error can't find the container with id fd0f5243dece3cdfb9cec1db67602c2cff2238ac73101e07d158fd81f0cdd319
Sep 19 10:52:12.280024 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:12.280000    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn" event={"ID":"70e95713-3c73-468c-ae4d-bf785a758838","Type":"ContainerStarted","Data":"fd0f5243dece3cdfb9cec1db67602c2cff2238ac73101e07d158fd81f0cdd319"}
Sep 19 10:52:13.287438 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:13.287406    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn" event={"ID":"70e95713-3c73-468c-ae4d-bf785a758838","Type":"ContainerStarted","Data":"56f682689be34f6860d16fd8b5be01fdb26d2800306e90af6eb526661158c9ed"}
Sep 19 10:52:13.304109 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:13.304055    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn" podStartSLOduration=2.304040325 podStartE2EDuration="2.304040325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:52:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:52:13.302108833 +0000 UTC m=+1054.213802352" watchObservedRunningTime="2026-09-19 10:52:13.304040325 +0000 UTC m=+1054.215733843"
Sep 19 10:52:14.402387 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:14.402357    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:52:19.256316 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:19.256289    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:19.863781 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:19.863747    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:20.468472 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:20.468436    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:21.165945 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:21.165910    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:21.766716 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:21.766675    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:21.952677 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:21.952645    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:21.952879 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:21.952696    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:21.965054 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:21.965024    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:22.347212 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:22.347178    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:52:22.352185 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:22.352159    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:22.480189 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:22.480157    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:22.582239 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:22.582202    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:23.550774 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:23.550695    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:24.157200 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:24.157171    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:24.258253 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:24.258222    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:24.584493 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:24.584465    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:30.538900 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:30.538872    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:30.597038 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:30.596994    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:31.665123 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:31.665096    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:32.194023 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:32.193987    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:32.347324 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:32.347294    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"]
Sep 19 10:52:32.364663 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:32.364630    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:32.761726 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:32.761693    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:33.476267 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:33.476237    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:34.086599 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:34.086569    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:34.223414 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:34.223382    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"]
Sep 19 10:52:34.242514 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:34.242490    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:34.866661 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:34.866619    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:35.578288 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:35.578257    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:36.214427 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:36.214396    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:36.946491 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:36.946463    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:37.559983 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:37.559941    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:38.239765 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:38.239732    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:41.031274 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:41.031235    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:41.547212 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:41.547175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:41.608387 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:41.608356    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:41.664305 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:41.664271    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:41.718919 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:41.718868    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:41.777032 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:41.777004    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:42.055708 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:42.055664    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:42.157458 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:42.157429    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:42.770776 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:42.770742    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:42.893031 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:42.893001    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:52:43.448705 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:43.448673    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:44.161031 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:44.161001    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:44.875162 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:44.875124    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:45.390021 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:45.389969    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:45.489241 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:45.489200    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:45.578181 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:45.578144    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:45.630407 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:45.630366    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:45.914514 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:45.914481    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:46.344671 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:46.344640    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:46.971265 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:46.971237    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:47.368680 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.368591    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k" podUID="a3b47f44-d649-44c6-ae56-e92a67c29516" containerName="main" containerID="cri-o://44999f4bcbbc1aa035b57dd187644ae9891ea3de3dee573017926217843d4486" gracePeriod=45
Sep 19 10:52:47.605086 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.605065    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:52:47.671904 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.671837    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:47.676997 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.676973    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-home\") pod \"a3b47f44-d649-44c6-ae56-e92a67c29516\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") "
Sep 19 10:52:47.677110 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.677009    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fq2kn\" (UniqueName: \"kubernetes.io/projected/a3b47f44-d649-44c6-ae56-e92a67c29516-kube-api-access-fq2kn\") pod \"a3b47f44-d649-44c6-ae56-e92a67c29516\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") "
Sep 19 10:52:47.677110 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.677052    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-tmp-dir\") pod \"a3b47f44-d649-44c6-ae56-e92a67c29516\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") "
Sep 19 10:52:47.677110 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.677104    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-model-cache\") pod \"a3b47f44-d649-44c6-ae56-e92a67c29516\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") "
Sep 19 10:52:47.677272 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.677159    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-dshm\") pod \"a3b47f44-d649-44c6-ae56-e92a67c29516\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") "
Sep 19 10:52:47.677272 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.677182    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b47f44-d649-44c6-ae56-e92a67c29516-tls-certs\") pod \"a3b47f44-d649-44c6-ae56-e92a67c29516\" (UID: \"a3b47f44-d649-44c6-ae56-e92a67c29516\") "
Sep 19 10:52:47.677272 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.677207    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-home" (OuterVolumeSpecName: "home") pod "a3b47f44-d649-44c6-ae56-e92a67c29516" (UID: "a3b47f44-d649-44c6-ae56-e92a67c29516"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:52:47.677433 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.677329    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a3b47f44-d649-44c6-ae56-e92a67c29516" (UID: "a3b47f44-d649-44c6-ae56-e92a67c29516"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:52:47.677433 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.677394    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-model-cache" (OuterVolumeSpecName: "model-cache") pod "a3b47f44-d649-44c6-ae56-e92a67c29516" (UID: "a3b47f44-d649-44c6-ae56-e92a67c29516"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:52:47.677556 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.677481    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:52:47.677556 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.677500    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:52:47.679337 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.679312    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-dshm" (OuterVolumeSpecName: "dshm") pod "a3b47f44-d649-44c6-ae56-e92a67c29516" (UID: "a3b47f44-d649-44c6-ae56-e92a67c29516"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:52:47.679337 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.679329    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b47f44-d649-44c6-ae56-e92a67c29516-kube-api-access-fq2kn" (OuterVolumeSpecName: "kube-api-access-fq2kn") pod "a3b47f44-d649-44c6-ae56-e92a67c29516" (UID: "a3b47f44-d649-44c6-ae56-e92a67c29516"). InnerVolumeSpecName "kube-api-access-fq2kn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:52:47.679477 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.679341    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3b47f44-d649-44c6-ae56-e92a67c29516-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a3b47f44-d649-44c6-ae56-e92a67c29516" (UID: "a3b47f44-d649-44c6-ae56-e92a67c29516"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:52:47.778394 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.778352    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:52:47.778394 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.778385    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3b47f44-d649-44c6-ae56-e92a67c29516-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:52:47.778394 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.778394    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3b47f44-d649-44c6-ae56-e92a67c29516-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:52:47.778394 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:47.778402    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fq2kn\" (UniqueName: \"kubernetes.io/projected/a3b47f44-d649-44c6-ae56-e92a67c29516-kube-api-access-fq2kn\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:52:48.303775 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:48.303741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:48.403765 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:48.403732    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:48.479045 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:48.479017    2579 generic.go:356] "Generic (PLEG): container finished" podID="a3b47f44-d649-44c6-ae56-e92a67c29516" containerID="44999f4bcbbc1aa035b57dd187644ae9891ea3de3dee573017926217843d4486" exitCode=0
Sep 19 10:52:48.479187 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:48.479057    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"
Sep 19 10:52:48.479187 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:48.479098    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k" event={"ID":"a3b47f44-d649-44c6-ae56-e92a67c29516","Type":"ContainerDied","Data":"44999f4bcbbc1aa035b57dd187644ae9891ea3de3dee573017926217843d4486"}
Sep 19 10:52:48.479187 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:48.479135    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k" event={"ID":"a3b47f44-d649-44c6-ae56-e92a67c29516","Type":"ContainerDied","Data":"4dd37ac0a57cd924ec9c8deaca63556f9fd7951b6277f80a33b97491c87e6580"}
Sep 19 10:52:48.479187 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:48.479154    2579 scope.go:117] "RemoveContainer" containerID="44999f4bcbbc1aa035b57dd187644ae9891ea3de3dee573017926217843d4486"
Sep 19 10:52:48.491115 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:48.491097    2579 scope.go:117] "RemoveContainer" containerID="44999f4bcbbc1aa035b57dd187644ae9891ea3de3dee573017926217843d4486"
Sep 19 10:52:48.491390 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:52:48.491368    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44999f4bcbbc1aa035b57dd187644ae9891ea3de3dee573017926217843d4486\": container with ID starting with 44999f4bcbbc1aa035b57dd187644ae9891ea3de3dee573017926217843d4486 not found: ID does not exist" containerID="44999f4bcbbc1aa035b57dd187644ae9891ea3de3dee573017926217843d4486"
Sep 19 10:52:48.491459 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:48.491401    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44999f4bcbbc1aa035b57dd187644ae9891ea3de3dee573017926217843d4486"} err="failed to get container status \"44999f4bcbbc1aa035b57dd187644ae9891ea3de3dee573017926217843d4486\": rpc error: code = NotFound desc = could not find container \"44999f4bcbbc1aa035b57dd187644ae9891ea3de3dee573017926217843d4486\": container with ID starting with 44999f4bcbbc1aa035b57dd187644ae9891ea3de3dee573017926217843d4486 not found: ID does not exist"
Sep 19 10:52:48.505866 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:48.505841    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"]
Sep 19 10:52:48.514436 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:48.514411    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-f91f97/test-model-f91f97-kserve-6859df89c7-z6q5k"]
Sep 19 10:52:48.548766 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:48.548742    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:48.853407 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:48.853377    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:48.949268 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:48.949238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:49.049908 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.049878    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:49.115241 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.115157    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3b47f44-d649-44c6-ae56-e92a67c29516" path="/var/lib/kubelet/pods/a3b47f44-d649-44c6-ae56-e92a67c29516/volumes"
Sep 19 10:52:49.242670 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.242632    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf" podUID="a2cf1c1a-2156-470b-9248-fcfa21685809" containerName="main" containerID="cri-o://6b8e3849efec88c091a74edb2fc4d22c1e41e9f702cebe6da013a221432953fc" gracePeriod=45
Sep 19 10:52:49.486858 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.486825    2579 generic.go:356] "Generic (PLEG): container finished" podID="a2cf1c1a-2156-470b-9248-fcfa21685809" containerID="6b8e3849efec88c091a74edb2fc4d22c1e41e9f702cebe6da013a221432953fc" exitCode=0
Sep 19 10:52:49.487150 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.486888    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf" event={"ID":"a2cf1c1a-2156-470b-9248-fcfa21685809","Type":"ContainerDied","Data":"6b8e3849efec88c091a74edb2fc4d22c1e41e9f702cebe6da013a221432953fc"}
Sep 19 10:52:49.487150 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.486923    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf" event={"ID":"a2cf1c1a-2156-470b-9248-fcfa21685809","Type":"ContainerDied","Data":"7e9007200c9910fb9e42e1ad1609586159904c40a1fff1a74350d3e84ef94c74"}
Sep 19 10:52:49.487150 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.486936    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e9007200c9910fb9e42e1ad1609586159904c40a1fff1a74350d3e84ef94c74"
Sep 19 10:52:49.496668 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.496650    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:52:49.596305 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.596280    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlmkq\" (UniqueName: \"kubernetes.io/projected/a2cf1c1a-2156-470b-9248-fcfa21685809-kube-api-access-mlmkq\") pod \"a2cf1c1a-2156-470b-9248-fcfa21685809\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") "
Sep 19 10:52:49.596424 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.596324    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-dshm\") pod \"a2cf1c1a-2156-470b-9248-fcfa21685809\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") "
Sep 19 10:52:49.596424 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.596364    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-home\") pod \"a2cf1c1a-2156-470b-9248-fcfa21685809\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") "
Sep 19 10:52:49.596424 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.596386    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2cf1c1a-2156-470b-9248-fcfa21685809-tls-certs\") pod \"a2cf1c1a-2156-470b-9248-fcfa21685809\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") "
Sep 19 10:52:49.596584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.596454    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-model-cache\") pod \"a2cf1c1a-2156-470b-9248-fcfa21685809\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") "
Sep 19 10:52:49.596650 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.596593    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-tmp-dir\") pod \"a2cf1c1a-2156-470b-9248-fcfa21685809\" (UID: \"a2cf1c1a-2156-470b-9248-fcfa21685809\") "
Sep 19 10:52:49.596710 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.596685    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-home" (OuterVolumeSpecName: "home") pod "a2cf1c1a-2156-470b-9248-fcfa21685809" (UID: "a2cf1c1a-2156-470b-9248-fcfa21685809"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:52:49.596761 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.596701    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-model-cache" (OuterVolumeSpecName: "model-cache") pod "a2cf1c1a-2156-470b-9248-fcfa21685809" (UID: "a2cf1c1a-2156-470b-9248-fcfa21685809"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:52:49.596882 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.596856    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a2cf1c1a-2156-470b-9248-fcfa21685809" (UID: "a2cf1c1a-2156-470b-9248-fcfa21685809"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:52:49.597005 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.596977    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:52:49.597005 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.596999    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:52:49.597123 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.597012    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:52:49.598516 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.598492    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2cf1c1a-2156-470b-9248-fcfa21685809-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a2cf1c1a-2156-470b-9248-fcfa21685809" (UID: "a2cf1c1a-2156-470b-9248-fcfa21685809"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:52:49.598637 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.598559    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-dshm" (OuterVolumeSpecName: "dshm") pod "a2cf1c1a-2156-470b-9248-fcfa21685809" (UID: "a2cf1c1a-2156-470b-9248-fcfa21685809"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:52:49.598693 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.598629    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2cf1c1a-2156-470b-9248-fcfa21685809-kube-api-access-mlmkq" (OuterVolumeSpecName: "kube-api-access-mlmkq") pod "a2cf1c1a-2156-470b-9248-fcfa21685809" (UID: "a2cf1c1a-2156-470b-9248-fcfa21685809"). InnerVolumeSpecName "kube-api-access-mlmkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:52:49.698408 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.698383    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlmkq\" (UniqueName: \"kubernetes.io/projected/a2cf1c1a-2156-470b-9248-fcfa21685809-kube-api-access-mlmkq\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:52:49.698408 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.698404    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2cf1c1a-2156-470b-9248-fcfa21685809-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:52:49.698578 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:49.698414    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2cf1c1a-2156-470b-9248-fcfa21685809-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:52:50.492021 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:50.491981    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"
Sep 19 10:52:50.512967 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:50.512945    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"]
Sep 19 10:52:50.516421 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:50.516400    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-6cf6a0/test-model-6cf6a0-kserve-59cbd7c765-s4phf"]
Sep 19 10:52:51.115876 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:51.115846    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2cf1c1a-2156-470b-9248-fcfa21685809" path="/var/lib/kubelet/pods/a2cf1c1a-2156-470b-9248-fcfa21685809/volumes"
Sep 19 10:52:51.583559 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:51.583511    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:51.647710 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:51.647683    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:51.828178 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:51.828153    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:52.294898 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:52.294869    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:53.006698 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:53.006668    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:53.742358 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:53.742331    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:54.258806 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:54.258775    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:54.857724 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:54.857668    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:55.402696 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:55.402663    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:55.461292 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:55.461261    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:55.538184 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:55.538157    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:55.636363 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:55.636339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:55.739151 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:55.739126    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:52:58.960600 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:52:58.960569    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:01.080713 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:01.080684    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:01.908766 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:01.908735    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:02.009197 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:02.009173    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:02.280777 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:02.280747    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:02.775731 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:02.775691    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"]
Sep 19 10:53:02.775985 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:02.775957    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79" podUID="ef05874d-aa6b-411a-976f-6fa71e5328cf" containerName="maas-api" containerID="cri-o://10860d40feaae026a2dd489b0d99b9e5ff99712762988dabe0fc7fd23de7f30c" gracePeriod=30
Sep 19 10:53:02.803870 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:02.803827    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:02.880774 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:02.880741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:02.938486 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:02.938448    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:03.023837 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.023808    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:53:03.031928 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.031863    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:03.094184 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.094148    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:03.123212 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.123181    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef05874d-aa6b-411a-976f-6fa71e5328cf-maas-api-tls\") pod \"ef05874d-aa6b-411a-976f-6fa71e5328cf\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") "
Sep 19 10:53:03.123385 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.123247    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9n8mj\" (UniqueName: \"kubernetes.io/projected/ef05874d-aa6b-411a-976f-6fa71e5328cf-kube-api-access-9n8mj\") pod \"ef05874d-aa6b-411a-976f-6fa71e5328cf\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") "
Sep 19 10:53:03.123385 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.123357    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef05874d-aa6b-411a-976f-6fa71e5328cf-metrics-certs\") pod \"ef05874d-aa6b-411a-976f-6fa71e5328cf\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") "
Sep 19 10:53:03.123513 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.123390    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef05874d-aa6b-411a-976f-6fa71e5328cf-openshift-service-ca\") pod \"ef05874d-aa6b-411a-976f-6fa71e5328cf\" (UID: \"ef05874d-aa6b-411a-976f-6fa71e5328cf\") "
Sep 19 10:53:03.123827 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.123796    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef05874d-aa6b-411a-976f-6fa71e5328cf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ef05874d-aa6b-411a-976f-6fa71e5328cf" (UID: "ef05874d-aa6b-411a-976f-6fa71e5328cf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:53:03.125669 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.125646    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef05874d-aa6b-411a-976f-6fa71e5328cf-kube-api-access-9n8mj" (OuterVolumeSpecName: "kube-api-access-9n8mj") pod "ef05874d-aa6b-411a-976f-6fa71e5328cf" (UID: "ef05874d-aa6b-411a-976f-6fa71e5328cf"). InnerVolumeSpecName "kube-api-access-9n8mj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:53:03.125790 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.125702    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef05874d-aa6b-411a-976f-6fa71e5328cf-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ef05874d-aa6b-411a-976f-6fa71e5328cf" (UID: "ef05874d-aa6b-411a-976f-6fa71e5328cf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:53:03.125790 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.125769    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef05874d-aa6b-411a-976f-6fa71e5328cf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ef05874d-aa6b-411a-976f-6fa71e5328cf" (UID: "ef05874d-aa6b-411a-976f-6fa71e5328cf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:53:03.175007 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.174967    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:03.224989 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.224959    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef05874d-aa6b-411a-976f-6fa71e5328cf-metrics-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:53:03.224989 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.224983    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef05874d-aa6b-411a-976f-6fa71e5328cf-openshift-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:53:03.224989 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.224994    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef05874d-aa6b-411a-976f-6fa71e5328cf-maas-api-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:53:03.225215 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.225005    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9n8mj\" (UniqueName: \"kubernetes.io/projected/ef05874d-aa6b-411a-976f-6fa71e5328cf-kube-api-access-9n8mj\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:53:03.376911 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.376824    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:03.477699 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.477671    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:03.562664 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.562633    2579 generic.go:356] "Generic (PLEG): container finished" podID="ef05874d-aa6b-411a-976f-6fa71e5328cf" containerID="10860d40feaae026a2dd489b0d99b9e5ff99712762988dabe0fc7fd23de7f30c" exitCode=0
Sep 19 10:53:03.562664 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.562667    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79" event={"ID":"ef05874d-aa6b-411a-976f-6fa71e5328cf","Type":"ContainerDied","Data":"10860d40feaae026a2dd489b0d99b9e5ff99712762988dabe0fc7fd23de7f30c"}
Sep 19 10:53:03.562890 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.562688    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"
Sep 19 10:53:03.562890 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.562703    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79" event={"ID":"ef05874d-aa6b-411a-976f-6fa71e5328cf","Type":"ContainerDied","Data":"abfe3c842b35bee999b6c9a9cebed5f21f6704de6dfe5d9c992541c871c431a3"}
Sep 19 10:53:03.562890 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.562724    2579 scope.go:117] "RemoveContainer" containerID="10860d40feaae026a2dd489b0d99b9e5ff99712762988dabe0fc7fd23de7f30c"
Sep 19 10:53:03.572734 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.572706    2579 scope.go:117] "RemoveContainer" containerID="10860d40feaae026a2dd489b0d99b9e5ff99712762988dabe0fc7fd23de7f30c"
Sep 19 10:53:03.572984 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:53:03.572966    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10860d40feaae026a2dd489b0d99b9e5ff99712762988dabe0fc7fd23de7f30c\": container with ID starting with 10860d40feaae026a2dd489b0d99b9e5ff99712762988dabe0fc7fd23de7f30c not found: ID does not exist" containerID="10860d40feaae026a2dd489b0d99b9e5ff99712762988dabe0fc7fd23de7f30c"
Sep 19 10:53:03.573047 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.572992    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10860d40feaae026a2dd489b0d99b9e5ff99712762988dabe0fc7fd23de7f30c"} err="failed to get container status \"10860d40feaae026a2dd489b0d99b9e5ff99712762988dabe0fc7fd23de7f30c\": rpc error: code = NotFound desc = could not find container \"10860d40feaae026a2dd489b0d99b9e5ff99712762988dabe0fc7fd23de7f30c\": container with ID starting with 10860d40feaae026a2dd489b0d99b9e5ff99712762988dabe0fc7fd23de7f30c not found: ID does not exist"
Sep 19 10:53:03.583962 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.583943    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"]
Sep 19 10:53:03.588105 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.588085    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6cf6a0-6ccb5bc946-wlp79"]
Sep 19 10:53:03.709596 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.709563    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:03.973230 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:03.973135    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:04.586650 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:04.586611    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:05.120145 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:05.120101    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef05874d-aa6b-411a-976f-6fa71e5328cf" path="/var/lib/kubelet/pods/ef05874d-aa6b-411a-976f-6fa71e5328cf/volumes"
Sep 19 10:53:05.462923 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:05.462879    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:05.960783 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:05.960753    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:06.059170 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:06.059141    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:06.990779 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:06.990747    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:07.685121 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:07.685092    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:07.785868 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:07.785835    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:07.930204 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:07.930175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:08.127650 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:08.127577    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:08.226150 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:08.226120    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:09.244793 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:09.244761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:09.302268 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:09.302238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:09.488021 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:09.487994    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:10.009900 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:10.009873    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:10.287083 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:10.287011    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:10.547675 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:10.547607    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:12.865597 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:12.865565    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:13.043597 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:13.043560    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:13.135217 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:13.135144    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:13.660430 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:13.660394    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:14.281923 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:14.281890    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:15.037628 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.037595    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:15.388867 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.388793    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:53:15.748977 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.748949    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"]
Sep 19 10:53:15.749332 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.749308    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3b47f44-d649-44c6-ae56-e92a67c29516" containerName="main"
Sep 19 10:53:15.749332 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.749329    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3b47f44-d649-44c6-ae56-e92a67c29516" containerName="main"
Sep 19 10:53:15.749556 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.749353    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2cf1c1a-2156-470b-9248-fcfa21685809" containerName="main"
Sep 19 10:53:15.749556 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.749362    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2cf1c1a-2156-470b-9248-fcfa21685809" containerName="main"
Sep 19 10:53:15.749556 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.749373    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef05874d-aa6b-411a-976f-6fa71e5328cf" containerName="maas-api"
Sep 19 10:53:15.749556 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.749378    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef05874d-aa6b-411a-976f-6fa71e5328cf" containerName="maas-api"
Sep 19 10:53:15.749556 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.749472    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2cf1c1a-2156-470b-9248-fcfa21685809" containerName="main"
Sep 19 10:53:15.749556 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.749492    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef05874d-aa6b-411a-976f-6fa71e5328cf" containerName="maas-api"
Sep 19 10:53:15.749556 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.749506    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3b47f44-d649-44c6-ae56-e92a67c29516" containerName="main"
Sep 19 10:53:15.754107 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.754086    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:15.756584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.756564    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-72e262-serving-cert\""
Sep 19 10:53:15.763842 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.763814    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"]
Sep 19 10:53:15.792556 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.792534    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:15.834823 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.834798    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fec2d06-6168-4622-a51e-d2417ee14d5f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:15.834930 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.834848    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:15.834930 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.834869    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:15.835016 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.834955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-splwd\" (UniqueName: \"kubernetes.io/projected/0fec2d06-6168-4622-a51e-d2417ee14d5f-kube-api-access-splwd\") pod \"maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:15.898025 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.898000    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:15.936651 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.936628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:15.936755 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.936708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-splwd\" (UniqueName: \"kubernetes.io/projected/0fec2d06-6168-4622-a51e-d2417ee14d5f-kube-api-access-splwd\") pod \"maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:15.936819 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.936766    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fec2d06-6168-4622-a51e-d2417ee14d5f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:15.936871 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.936819    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:15.936929 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:53:15.936910    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-serving-cert: secret "maas-api-e2e-shared-a-w2-72e262-serving-cert" not found
Sep 19 10:53:15.936979 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:53:15.936966    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-maas-api-tls podName:0fec2d06-6168-4622-a51e-d2417ee14d5f nodeName:}" failed. No retries permitted until 2026-09-19 10:53:16.436948713 +0000 UTC m=+1117.348642227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-maas-api-tls") pod "maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4" (UID: "0fec2d06-6168-4622-a51e-d2417ee14d5f") : secret "maas-api-e2e-shared-a-w2-72e262-serving-cert" not found
Sep 19 10:53:15.937454 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.937431    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fec2d06-6168-4622-a51e-d2417ee14d5f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:15.939165 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.939149    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:15.954412 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:15.954390    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-splwd\" (UniqueName: \"kubernetes.io/projected/0fec2d06-6168-4622-a51e-d2417ee14d5f-kube-api-access-splwd\") pod \"maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:16.327252 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:16.327220    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:16.425497 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:16.425470    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:16.441767 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:16.441740    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:16.444053 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:16.444028    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:16.486395 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:16.486363    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:16.667103 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:16.667020    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:16.763646 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:16.763620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:16.938512 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:16.938482    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"]
Sep 19 10:53:16.940170 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:53:16.940143    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fec2d06_6168_4622_a51e_d2417ee14d5f.slice/crio-0a84ba88b0bfba8219750f9213048eebf71ca23c96f3cb7cbf596a111fb4837f WatchSource:0}: Error finding container 0a84ba88b0bfba8219750f9213048eebf71ca23c96f3cb7cbf596a111fb4837f: Status 404 returned error can't find the container with id 0a84ba88b0bfba8219750f9213048eebf71ca23c96f3cb7cbf596a111fb4837f
Sep 19 10:53:16.941515 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:16.941493    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 10:53:17.642334 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:17.642250    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4" event={"ID":"0fec2d06-6168-4622-a51e-d2417ee14d5f","Type":"ContainerStarted","Data":"3177957f6ca836b6e393a3983dbb7b13f3d94945a40369acd8874b10699e15c4"}
Sep 19 10:53:17.642334 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:17.642300    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4" event={"ID":"0fec2d06-6168-4622-a51e-d2417ee14d5f","Type":"ContainerStarted","Data":"0a84ba88b0bfba8219750f9213048eebf71ca23c96f3cb7cbf596a111fb4837f"}
Sep 19 10:53:17.642738 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:17.642339    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:17.659636 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:17.659593    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4" podStartSLOduration=2.330299662 podStartE2EDuration="2.659579762s" totalImagesPullingTime="329.2801ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:53:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:53:17.658738641 +0000 UTC m=+1118.570432160" watchObservedRunningTime="2026-09-19 10:53:17.659579762 +0000 UTC m=+1118.571273320"
Sep 19 10:53:17.674088 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:17.674064    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:23.653679 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:23.653649    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 10:53:29.065759 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:29.065728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:30.204762 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:30.204728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:30.883472 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:30.883434    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:31.070057 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:31.070028    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:31.206574 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:31.206545    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:31.593485 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:31.593415    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:31.764844 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:31.764811    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:31.866347 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:31.866248    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:32.440403 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:32.440370    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:32.871696 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:32.871600    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:33.072792 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:33.072765    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:33.173777 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:33.173748    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:33.461290 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:33.461214    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:33.695150 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:33.695121    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:33.800189 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:33.800110    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:34.603947 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:34.603920    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:35.141697 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:35.141667    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:35.269516 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:35.269485    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:35.364852 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:35.364826    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:35.988378 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:35.988351    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:36.771506 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:36.771476    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:36.969120 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:36.969078    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:37.067320 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:37.067238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:37.514553 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:37.514511    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:38.180835 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:38.180789    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:38.281312 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:38.281273    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:38.404690 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:38.404662    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:38.694945 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:38.694917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:38.795906 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:38.795875    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:39.258124 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.258089    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"]
Sep 19 10:53:39.261144 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.261124    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:39.263616 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.263588    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-99b43e-serving-cert\""
Sep 19 10:53:39.272953 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.272933    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"]
Sep 19 10:53:39.289208 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.289184    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:39.329710 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.329688    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7bd7059-22d0-468f-a405-cb5ee2056c37-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:39.329798 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.329726    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s5gx\" (UniqueName: \"kubernetes.io/projected/e7bd7059-22d0-468f-a405-cb5ee2056c37-kube-api-access-6s5gx\") pod \"maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:39.329798 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.329756    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:39.329872 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.329847    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:39.352212 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.352189    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:39.430341 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.430318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:39.430428 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.430372    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7bd7059-22d0-468f-a405-cb5ee2056c37-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:39.430428 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.430403    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s5gx\" (UniqueName: \"kubernetes.io/projected/e7bd7059-22d0-468f-a405-cb5ee2056c37-kube-api-access-6s5gx\") pod \"maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:39.430545 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.430428    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:39.430608 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:53:39.430592    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-serving-cert: secret "maas-api-e2e-shared-b-w2-99b43e-serving-cert" not found
Sep 19 10:53:39.430668 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:53:39.430658    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-maas-api-tls podName:e7bd7059-22d0-468f-a405-cb5ee2056c37 nodeName:}" failed. No retries permitted until 2026-09-19 10:53:39.930639546 +0000 UTC m=+1140.842333044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-maas-api-tls") pod "maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p" (UID: "e7bd7059-22d0-468f-a405-cb5ee2056c37") : secret "maas-api-e2e-shared-b-w2-99b43e-serving-cert" not found
Sep 19 10:53:39.431103 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.431086    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7bd7059-22d0-468f-a405-cb5ee2056c37-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:39.432756 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.432737    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:39.438443 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.438423    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s5gx\" (UniqueName: \"kubernetes.io/projected/e7bd7059-22d0-468f-a405-cb5ee2056c37-kube-api-access-6s5gx\") pod \"maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:39.662340 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.662268    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:39.817290 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.817261    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:39.918033 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.917949    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:39.936262 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.936230    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:39.938958 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:39.938938    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:40.172577 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:40.172550    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:40.268501 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:40.268476    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:40.445974 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:40.445952    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"]
Sep 19 10:53:40.448118 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:53:40.448092    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7bd7059_22d0_468f_a405_cb5ee2056c37.slice/crio-34235a6d25a62f4712ec8b84d875539e6f14d0354f9b85a6fbf45f622b787887 WatchSource:0}: Error finding container 34235a6d25a62f4712ec8b84d875539e6f14d0354f9b85a6fbf45f622b787887: Status 404 returned error can't find the container with id 34235a6d25a62f4712ec8b84d875539e6f14d0354f9b85a6fbf45f622b787887
Sep 19 10:53:40.765992 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:40.765907    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p" event={"ID":"e7bd7059-22d0-468f-a405-cb5ee2056c37","Type":"ContainerStarted","Data":"34235a6d25a62f4712ec8b84d875539e6f14d0354f9b85a6fbf45f622b787887"}
Sep 19 10:53:40.877309 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:40.877284    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:41.396458 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:41.396431    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:41.772604 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:41.772565    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p" event={"ID":"e7bd7059-22d0-468f-a405-cb5ee2056c37","Type":"ContainerStarted","Data":"590b8f91bbf90c581afd4dca84679a52c74a5c3331abb21b83d2cf6a4c72a71d"}
Sep 19 10:53:41.772806 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:41.772696    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:41.787880 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:41.787833    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p" podStartSLOduration=2.37509631 podStartE2EDuration="2.787821774s" totalImagesPullingTime="412.725464ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:53:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:53:41.787485523 +0000 UTC m=+1142.699179043" watchObservedRunningTime="2026-09-19 10:53:41.787821774 +0000 UTC m=+1142.699515292"
Sep 19 10:53:42.274261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:42.274222    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:42.969860 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:42.969826    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:43.758943 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:43.758913    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:44.446301 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:44.446270    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:45.165466 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:45.165436    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:53:47.783537 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:47.783491    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 10:53:52.299916 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:53:52.299871    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:54:03.691456 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:03.691384    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:05.088143 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:05.088108    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:05.280150 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:05.280116    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:05.886101 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:05.886073    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:06.019211 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:06.019177    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9074-38d395"]
Sep 19 10:54:06.022667 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:06.022645    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-38d395"
Sep 19 10:54:06.025315 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:06.025286    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9d6w5\""
Sep 19 10:54:06.027671 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:06.027648    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9074-38d395"]
Sep 19 10:54:06.070953 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:06.070927    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8mvp\" (UniqueName: \"kubernetes.io/projected/bdda993d-377d-404b-8359-ac16226c2bdf-kube-api-access-z8mvp\") pod \"test-curl-9074-38d395\" (UID: \"bdda993d-377d-404b-8359-ac16226c2bdf\") " pod="opendatahub/test-curl-9074-38d395"
Sep 19 10:54:06.172233 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:06.172165    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8mvp\" (UniqueName: \"kubernetes.io/projected/bdda993d-377d-404b-8359-ac16226c2bdf-kube-api-access-z8mvp\") pod \"test-curl-9074-38d395\" (UID: \"bdda993d-377d-404b-8359-ac16226c2bdf\") " pod="opendatahub/test-curl-9074-38d395"
Sep 19 10:54:06.180388 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:06.180363    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8mvp\" (UniqueName: \"kubernetes.io/projected/bdda993d-377d-404b-8359-ac16226c2bdf-kube-api-access-z8mvp\") pod \"test-curl-9074-38d395\" (UID: \"bdda993d-377d-404b-8359-ac16226c2bdf\") " pod="opendatahub/test-curl-9074-38d395"
Sep 19 10:54:06.334080 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:06.334054    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-38d395"
Sep 19 10:54:06.589251 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:06.589146    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:06.600776 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:06.600755    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9074-38d395"]
Sep 19 10:54:06.602204 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:54:06.602182    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbdda993d_377d_404b_8359_ac16226c2bdf.slice/crio-4842c949ebe19b1fb48fbe6b95343fe87003343c3acb5ffbbdd85a9fbac62a4a WatchSource:0}: Error finding container 4842c949ebe19b1fb48fbe6b95343fe87003343c3acb5ffbbdd85a9fbac62a4a: Status 404 returned error can't find the container with id 4842c949ebe19b1fb48fbe6b95343fe87003343c3acb5ffbbdd85a9fbac62a4a
Sep 19 10:54:06.909731 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:06.909625    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9074-38d395" event={"ID":"bdda993d-377d-404b-8359-ac16226c2bdf","Type":"ContainerStarted","Data":"66fd3deaadace64e35d8a3b47fad63d6ed3ae3b2fafeed87adc944da0ab1186c"}
Sep 19 10:54:06.909731 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:06.909676    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9074-38d395" event={"ID":"bdda993d-377d-404b-8359-ac16226c2bdf","Type":"ContainerStarted","Data":"4842c949ebe19b1fb48fbe6b95343fe87003343c3acb5ffbbdd85a9fbac62a4a"}
Sep 19 10:54:06.925020 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:06.924967    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-9074-38d395" podStartSLOduration=0.924950675 podStartE2EDuration="924.950675ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:54:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:54:06.922487188 +0000 UTC m=+1167.834180708" watchObservedRunningTime="2026-09-19 10:54:06.924950675 +0000 UTC m=+1167.836644197"
Sep 19 10:54:06.985003 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:06.984980    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:07.293425 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:07.293402    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:07.499779 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:07.499748    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9074-38d395"]
Sep 19 10:54:07.514691 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:07.514667    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9074-38d395"]
Sep 19 10:54:07.626211 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:07.626143    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9074-00ba26"]
Sep 19 10:54:07.626505 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:07.626493    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdda993d-377d-404b-8359-ac16226c2bdf" containerName="test-curl-9074-38d395"
Sep 19 10:54:07.626582 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:07.626507    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdda993d-377d-404b-8359-ac16226c2bdf" containerName="test-curl-9074-38d395"
Sep 19 10:54:07.626623 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:07.626614    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdda993d-377d-404b-8359-ac16226c2bdf" containerName="test-curl-9074-38d395"
Sep 19 10:54:07.629712 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:07.629696    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-00ba26"
Sep 19 10:54:07.641494 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:07.641466    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9074-00ba26"]
Sep 19 10:54:07.686975 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:07.686946    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62msq\" (UniqueName: \"kubernetes.io/projected/5572cebb-dc56-42c6-a7cf-3a2ef6caee1b-kube-api-access-62msq\") pod \"test-curl-9074-00ba26\" (UID: \"5572cebb-dc56-42c6-a7cf-3a2ef6caee1b\") " pod="opendatahub/test-curl-9074-00ba26"
Sep 19 10:54:07.788194 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:07.788162    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62msq\" (UniqueName: \"kubernetes.io/projected/5572cebb-dc56-42c6-a7cf-3a2ef6caee1b-kube-api-access-62msq\") pod \"test-curl-9074-00ba26\" (UID: \"5572cebb-dc56-42c6-a7cf-3a2ef6caee1b\") " pod="opendatahub/test-curl-9074-00ba26"
Sep 19 10:54:07.795604 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:07.795575    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62msq\" (UniqueName: \"kubernetes.io/projected/5572cebb-dc56-42c6-a7cf-3a2ef6caee1b-kube-api-access-62msq\") pod \"test-curl-9074-00ba26\" (UID: \"5572cebb-dc56-42c6-a7cf-3a2ef6caee1b\") " pod="opendatahub/test-curl-9074-00ba26"
Sep 19 10:54:07.915267 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:07.915200    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-9074-38d395" container="test-curl-9074-38d395"
Sep 19 10:54:07.939046 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:07.939017    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-00ba26"
Sep 19 10:54:08.208463 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:08.208436    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9074-00ba26"]
Sep 19 10:54:08.209758 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:54:08.209725    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5572cebb_dc56_42c6_a7cf_3a2ef6caee1b.slice/crio-4409e6a6f776d398dffd25200dba729baf857fe057008769a171fc823a6ffcb1 WatchSource:0}: Error finding container 4409e6a6f776d398dffd25200dba729baf857fe057008769a171fc823a6ffcb1: Status 404 returned error can't find the container with id 4409e6a6f776d398dffd25200dba729baf857fe057008769a171fc823a6ffcb1
Sep 19 10:54:08.255237 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:08.255212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:08.922137 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:08.922099    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9074-00ba26" event={"ID":"5572cebb-dc56-42c6-a7cf-3a2ef6caee1b","Type":"ContainerStarted","Data":"29453172ae8fb847721c3b907c488023eb5960640bc84038e644acbf8a248f46"}
Sep 19 10:54:08.922557 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:08.922151    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9074-00ba26" event={"ID":"5572cebb-dc56-42c6-a7cf-3a2ef6caee1b","Type":"ContainerStarted","Data":"4409e6a6f776d398dffd25200dba729baf857fe057008769a171fc823a6ffcb1"}
Sep 19 10:54:08.922557 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:08.922230    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-9074-38d395" podUID="bdda993d-377d-404b-8359-ac16226c2bdf" containerName="test-curl-9074-38d395" containerID="cri-o://66fd3deaadace64e35d8a3b47fad63d6ed3ae3b2fafeed87adc944da0ab1186c" gracePeriod=2
Sep 19 10:54:08.924236 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:08.924213    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9074-38d395\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="bdda993d-377d-404b-8359-ac16226c2bdf" pod="opendatahub/test-curl-9074-38d395"
Sep 19 10:54:08.944635 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:08.944601    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9074-38d395\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="bdda993d-377d-404b-8359-ac16226c2bdf" pod="opendatahub/test-curl-9074-38d395"
Sep 19 10:54:08.944966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:08.944934    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-9074-00ba26" podStartSLOduration=1.944922342 podStartE2EDuration="1.944922342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:54:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:54:08.942392909 +0000 UTC m=+1169.854086427" watchObservedRunningTime="2026-09-19 10:54:08.944922342 +0000 UTC m=+1169.856615861"
Sep 19 10:54:08.966550 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:08.966501    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:09.529570 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:09.529538    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9074-00ba26"]
Sep 19 10:54:09.534126 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:09.534091    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9074-00ba26"]
Sep 19 10:54:09.645206 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:09.645126    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9074-9e0d93"]
Sep 19 10:54:09.645597 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:09.645580    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5572cebb-dc56-42c6-a7cf-3a2ef6caee1b" containerName="test-curl-9074-00ba26"
Sep 19 10:54:09.645707 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:09.645600    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5572cebb-dc56-42c6-a7cf-3a2ef6caee1b" containerName="test-curl-9074-00ba26"
Sep 19 10:54:09.645707 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:09.645691    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5572cebb-dc56-42c6-a7cf-3a2ef6caee1b" containerName="test-curl-9074-00ba26"
Sep 19 10:54:09.648813 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:09.648794    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-9e0d93"
Sep 19 10:54:09.654708 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:09.654686    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9074-9e0d93"]
Sep 19 10:54:09.699433 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:09.699407    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:09.705669 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:09.705633    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd26n\" (UniqueName: \"kubernetes.io/projected/2d1b7d8a-815d-456c-92c1-15178550402c-kube-api-access-gd26n\") pod \"test-curl-9074-9e0d93\" (UID: \"2d1b7d8a-815d-456c-92c1-15178550402c\") " pod="opendatahub/test-curl-9074-9e0d93"
Sep 19 10:54:09.806393 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:09.806359    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd26n\" (UniqueName: \"kubernetes.io/projected/2d1b7d8a-815d-456c-92c1-15178550402c-kube-api-access-gd26n\") pod \"test-curl-9074-9e0d93\" (UID: \"2d1b7d8a-815d-456c-92c1-15178550402c\") " pod="opendatahub/test-curl-9074-9e0d93"
Sep 19 10:54:09.814073 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:09.814044    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd26n\" (UniqueName: \"kubernetes.io/projected/2d1b7d8a-815d-456c-92c1-15178550402c-kube-api-access-gd26n\") pod \"test-curl-9074-9e0d93\" (UID: \"2d1b7d8a-815d-456c-92c1-15178550402c\") " pod="opendatahub/test-curl-9074-9e0d93"
Sep 19 10:54:09.927993 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:09.927961    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-9074-00ba26" container="test-curl-9074-00ba26"
Sep 19 10:54:09.959963 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:09.959934    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-9e0d93"
Sep 19 10:54:10.229870 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:10.229845    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9074-9e0d93"]
Sep 19 10:54:10.230830 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:54:10.230800    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d1b7d8a_815d_456c_92c1_15178550402c.slice/crio-bceb381687bb54f8454f55f891686e462d341c5621936b6a551a57c86c5b9f75 WatchSource:0}: Error finding container bceb381687bb54f8454f55f891686e462d341c5621936b6a551a57c86c5b9f75: Status 404 returned error can't find the container with id bceb381687bb54f8454f55f891686e462d341c5621936b6a551a57c86c5b9f75
Sep 19 10:54:10.354059 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:10.354032    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:10.937065 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:10.937030    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9074-9e0d93" event={"ID":"2d1b7d8a-815d-456c-92c1-15178550402c","Type":"ContainerStarted","Data":"17d1a85391f67a1deabc6a02bbe16c51de99e2d7b4c50d10e18fd2fa27234cf4"}
Sep 19 10:54:10.937429 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:10.937079    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9074-9e0d93" event={"ID":"2d1b7d8a-815d-456c-92c1-15178550402c","Type":"ContainerStarted","Data":"bceb381687bb54f8454f55f891686e462d341c5621936b6a551a57c86c5b9f75"}
Sep 19 10:54:10.937429 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:10.937084    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-9074-00ba26" podUID="5572cebb-dc56-42c6-a7cf-3a2ef6caee1b" containerName="test-curl-9074-00ba26" containerID="cri-o://29453172ae8fb847721c3b907c488023eb5960640bc84038e644acbf8a248f46" gracePeriod=2
Sep 19 10:54:10.951326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:10.951282    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-9074-9e0d93" podStartSLOduration=1.9512672819999999 podStartE2EDuration="1.951267282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:54:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:54:10.9494636 +0000 UTC m=+1171.861157120" watchObservedRunningTime="2026-09-19 10:54:10.951267282 +0000 UTC m=+1171.862960800"
Sep 19 10:54:10.951428 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:10.951397    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9074-00ba26\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="5572cebb-dc56-42c6-a7cf-3a2ef6caee1b" pod="opendatahub/test-curl-9074-00ba26"
Sep 19 10:54:10.953337 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:10.953313    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9074-00ba26\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="5572cebb-dc56-42c6-a7cf-3a2ef6caee1b" pod="opendatahub/test-curl-9074-00ba26"
Sep 19 10:54:11.042171 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.042145    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-38d395"
Sep 19 10:54:11.044265 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:11.044243    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9074-00ba26\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="5572cebb-dc56-42c6-a7cf-3a2ef6caee1b" pod="opendatahub/test-curl-9074-00ba26"
Sep 19 10:54:11.063274 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.063253    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:11.114718 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:11.114688    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9074-00ba26\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="5572cebb-dc56-42c6-a7cf-3a2ef6caee1b" pod="opendatahub/test-curl-9074-00ba26"
Sep 19 10:54:11.117749 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.117726    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8mvp\" (UniqueName: \"kubernetes.io/projected/bdda993d-377d-404b-8359-ac16226c2bdf-kube-api-access-z8mvp\") pod \"bdda993d-377d-404b-8359-ac16226c2bdf\" (UID: \"bdda993d-377d-404b-8359-ac16226c2bdf\") "
Sep 19 10:54:11.119775 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.119751    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdda993d-377d-404b-8359-ac16226c2bdf-kube-api-access-z8mvp" (OuterVolumeSpecName: "kube-api-access-z8mvp") pod "bdda993d-377d-404b-8359-ac16226c2bdf" (UID: "bdda993d-377d-404b-8359-ac16226c2bdf"). InnerVolumeSpecName "kube-api-access-z8mvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:54:11.219613 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.219291    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8mvp\" (UniqueName: \"kubernetes.io/projected/bdda993d-377d-404b-8359-ac16226c2bdf-kube-api-access-z8mvp\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:54:11.493778 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.493699    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9074-9e0d93"]
Sep 19 10:54:11.497887 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.497859    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9074-9e0d93"]
Sep 19 10:54:11.609593 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.609561    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9074-98fe4e"]
Sep 19 10:54:11.609962 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.609948    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d1b7d8a-815d-456c-92c1-15178550402c" containerName="test-curl-9074-9e0d93"
Sep 19 10:54:11.610007 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.609964    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d1b7d8a-815d-456c-92c1-15178550402c" containerName="test-curl-9074-9e0d93"
Sep 19 10:54:11.610058 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.610049    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d1b7d8a-815d-456c-92c1-15178550402c" containerName="test-curl-9074-9e0d93"
Sep 19 10:54:11.613275 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.613250    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-98fe4e"
Sep 19 10:54:11.616839 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.616809    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9074-98fe4e"]
Sep 19 10:54:11.723841 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.723812    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4m2l\" (UniqueName: \"kubernetes.io/projected/abbb1fca-525e-44f5-94bd-58e30698300e-kube-api-access-w4m2l\") pod \"test-curl-9074-98fe4e\" (UID: \"abbb1fca-525e-44f5-94bd-58e30698300e\") " pod="opendatahub/test-curl-9074-98fe4e"
Sep 19 10:54:11.825234 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.825159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4m2l\" (UniqueName: \"kubernetes.io/projected/abbb1fca-525e-44f5-94bd-58e30698300e-kube-api-access-w4m2l\") pod \"test-curl-9074-98fe4e\" (UID: \"abbb1fca-525e-44f5-94bd-58e30698300e\") " pod="opendatahub/test-curl-9074-98fe4e"
Sep 19 10:54:11.833290 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.833264    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4m2l\" (UniqueName: \"kubernetes.io/projected/abbb1fca-525e-44f5-94bd-58e30698300e-kube-api-access-w4m2l\") pod \"test-curl-9074-98fe4e\" (UID: \"abbb1fca-525e-44f5-94bd-58e30698300e\") " pod="opendatahub/test-curl-9074-98fe4e"
Sep 19 10:54:11.923282 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.923254    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-98fe4e"
Sep 19 10:54:11.944508 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.944483    2579 generic.go:356] "Generic (PLEG): container finished" podID="bdda993d-377d-404b-8359-ac16226c2bdf" containerID="66fd3deaadace64e35d8a3b47fad63d6ed3ae3b2fafeed87adc944da0ab1186c" exitCode=137
Sep 19 10:54:11.944886 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.944574    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-38d395"
Sep 19 10:54:11.944886 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.944592    2579 scope.go:117] "RemoveContainer" containerID="66fd3deaadace64e35d8a3b47fad63d6ed3ae3b2fafeed87adc944da0ab1186c"
Sep 19 10:54:11.944886 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:11.944811    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-9074-9e0d93" container="test-curl-9074-9e0d93"
Sep 19 10:54:11.954411 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.954389    2579 scope.go:117] "RemoveContainer" containerID="66fd3deaadace64e35d8a3b47fad63d6ed3ae3b2fafeed87adc944da0ab1186c"
Sep 19 10:54:11.954734 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:11.954710    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66fd3deaadace64e35d8a3b47fad63d6ed3ae3b2fafeed87adc944da0ab1186c\": container with ID starting with 66fd3deaadace64e35d8a3b47fad63d6ed3ae3b2fafeed87adc944da0ab1186c not found: ID does not exist" containerID="66fd3deaadace64e35d8a3b47fad63d6ed3ae3b2fafeed87adc944da0ab1186c"
Sep 19 10:54:11.954836 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:11.954760    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66fd3deaadace64e35d8a3b47fad63d6ed3ae3b2fafeed87adc944da0ab1186c"} err="failed to get container status \"66fd3deaadace64e35d8a3b47fad63d6ed3ae3b2fafeed87adc944da0ab1186c\": rpc error: code = NotFound desc = could not find container \"66fd3deaadace64e35d8a3b47fad63d6ed3ae3b2fafeed87adc944da0ab1186c\": container with ID starting with 66fd3deaadace64e35d8a3b47fad63d6ed3ae3b2fafeed87adc944da0ab1186c not found: ID does not exist"
Sep 19 10:54:12.193034 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:12.193009    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9074-98fe4e"]
Sep 19 10:54:12.195027 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:54:12.195002    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabbb1fca_525e_44f5_94bd_58e30698300e.slice/crio-6365c2e982a08927b53321a7dd4aadfb0230ee469001777af5d62f5c8c802b68 WatchSource:0}: Error finding container 6365c2e982a08927b53321a7dd4aadfb0230ee469001777af5d62f5c8c802b68: Status 404 returned error can't find the container with id 6365c2e982a08927b53321a7dd4aadfb0230ee469001777af5d62f5c8c802b68
Sep 19 10:54:12.953867 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:12.953840    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9074-98fe4e" event={"ID":"abbb1fca-525e-44f5-94bd-58e30698300e","Type":"ContainerStarted","Data":"e253616deb02d0e916812aa4226c05679155c7d39d99cf14bd22a6cd37b4a87f"}
Sep 19 10:54:12.954194 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:12.953877    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9074-98fe4e" event={"ID":"abbb1fca-525e-44f5-94bd-58e30698300e","Type":"ContainerStarted","Data":"6365c2e982a08927b53321a7dd4aadfb0230ee469001777af5d62f5c8c802b68"}
Sep 19 10:54:12.954194 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:12.953926    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-9074-9e0d93" podUID="2d1b7d8a-815d-456c-92c1-15178550402c" containerName="test-curl-9074-9e0d93" containerID="cri-o://17d1a85391f67a1deabc6a02bbe16c51de99e2d7b4c50d10e18fd2fa27234cf4" gracePeriod=2
Sep 19 10:54:12.967889 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:12.967861    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9074-9e0d93\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="2d1b7d8a-815d-456c-92c1-15178550402c" pod="opendatahub/test-curl-9074-9e0d93"
Sep 19 10:54:12.968821 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:12.968783    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-9074-98fe4e" podStartSLOduration=1.968769877 podStartE2EDuration="1.968769877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:54:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:54:12.965950976 +0000 UTC m=+1173.877644496" watchObservedRunningTime="2026-09-19 10:54:12.968769877 +0000 UTC m=+1173.880463445"
Sep 19 10:54:12.969659 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:12.969639    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9074-9e0d93\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="2d1b7d8a-815d-456c-92c1-15178550402c" pod="opendatahub/test-curl-9074-9e0d93"
Sep 19 10:54:13.115984 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:13.115957    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdda993d-377d-404b-8359-ac16226c2bdf" path="/var/lib/kubelet/pods/bdda993d-377d-404b-8359-ac16226c2bdf/volumes"
Sep 19 10:54:13.123473 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:13.123452    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-00ba26"
Sep 19 10:54:13.237785 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:13.237724    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62msq\" (UniqueName: \"kubernetes.io/projected/5572cebb-dc56-42c6-a7cf-3a2ef6caee1b-kube-api-access-62msq\") pod \"5572cebb-dc56-42c6-a7cf-3a2ef6caee1b\" (UID: \"5572cebb-dc56-42c6-a7cf-3a2ef6caee1b\") "
Sep 19 10:54:13.239573 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:13.239534    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5572cebb-dc56-42c6-a7cf-3a2ef6caee1b-kube-api-access-62msq" (OuterVolumeSpecName: "kube-api-access-62msq") pod "5572cebb-dc56-42c6-a7cf-3a2ef6caee1b" (UID: "5572cebb-dc56-42c6-a7cf-3a2ef6caee1b"). InnerVolumeSpecName "kube-api-access-62msq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:54:13.338453 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:13.338430    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62msq\" (UniqueName: \"kubernetes.io/projected/5572cebb-dc56-42c6-a7cf-3a2ef6caee1b-kube-api-access-62msq\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:54:13.514892 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:13.514821    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9074-98fe4e"]
Sep 19 10:54:13.518759 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:13.518736    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9074-98fe4e"]
Sep 19 10:54:13.960166 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:13.960137    2579 generic.go:356] "Generic (PLEG): container finished" podID="5572cebb-dc56-42c6-a7cf-3a2ef6caee1b" containerID="29453172ae8fb847721c3b907c488023eb5960640bc84038e644acbf8a248f46" exitCode=137
Sep 19 10:54:13.960606 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:13.960175    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-00ba26"
Sep 19 10:54:13.960606 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:13.960236    2579 scope.go:117] "RemoveContainer" containerID="29453172ae8fb847721c3b907c488023eb5960640bc84038e644acbf8a248f46"
Sep 19 10:54:13.960749 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:13.960657    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-9074-98fe4e" container="test-curl-9074-98fe4e"
Sep 19 10:54:13.969434 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:13.969420    2579 scope.go:117] "RemoveContainer" containerID="29453172ae8fb847721c3b907c488023eb5960640bc84038e644acbf8a248f46"
Sep 19 10:54:13.969717 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:13.969697    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29453172ae8fb847721c3b907c488023eb5960640bc84038e644acbf8a248f46\": container with ID starting with 29453172ae8fb847721c3b907c488023eb5960640bc84038e644acbf8a248f46 not found: ID does not exist" containerID="29453172ae8fb847721c3b907c488023eb5960640bc84038e644acbf8a248f46"
Sep 19 10:54:13.969794 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:13.969728    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29453172ae8fb847721c3b907c488023eb5960640bc84038e644acbf8a248f46"} err="failed to get container status \"29453172ae8fb847721c3b907c488023eb5960640bc84038e644acbf8a248f46\": rpc error: code = NotFound desc = could not find container \"29453172ae8fb847721c3b907c488023eb5960640bc84038e644acbf8a248f46\": container with ID starting with 29453172ae8fb847721c3b907c488023eb5960640bc84038e644acbf8a248f46 not found: ID does not exist"
Sep 19 10:54:13.970233 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:13.970211    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9074-00ba26\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="5572cebb-dc56-42c6-a7cf-3a2ef6caee1b" pod="opendatahub/test-curl-9074-00ba26"
Sep 19 10:54:14.085641 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:14.085612    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:14.377465 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:14.377378    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:14.532853 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:14.532816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:14.966579 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:14.966550    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-9074-98fe4e" podUID="abbb1fca-525e-44f5-94bd-58e30698300e" containerName="test-curl-9074-98fe4e" containerID="cri-o://e253616deb02d0e916812aa4226c05679155c7d39d99cf14bd22a6cd37b4a87f" gracePeriod=2
Sep 19 10:54:14.968730 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:14.968699    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9074-00ba26\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="5572cebb-dc56-42c6-a7cf-3a2ef6caee1b" pod="opendatahub/test-curl-9074-00ba26"
Sep 19 10:54:14.970764 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:14.970734    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9074-98fe4e\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="abbb1fca-525e-44f5-94bd-58e30698300e" pod="opendatahub/test-curl-9074-98fe4e"
Sep 19 10:54:15.116679 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:15.116651    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5572cebb-dc56-42c6-a7cf-3a2ef6caee1b" path="/var/lib/kubelet/pods/5572cebb-dc56-42c6-a7cf-3a2ef6caee1b/volumes"
Sep 19 10:54:15.135662 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:15.135641    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-9e0d93"
Sep 19 10:54:15.263978 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:15.263897    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd26n\" (UniqueName: \"kubernetes.io/projected/2d1b7d8a-815d-456c-92c1-15178550402c-kube-api-access-gd26n\") pod \"2d1b7d8a-815d-456c-92c1-15178550402c\" (UID: \"2d1b7d8a-815d-456c-92c1-15178550402c\") "
Sep 19 10:54:15.265940 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:15.265915    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d1b7d8a-815d-456c-92c1-15178550402c-kube-api-access-gd26n" (OuterVolumeSpecName: "kube-api-access-gd26n") pod "2d1b7d8a-815d-456c-92c1-15178550402c" (UID: "2d1b7d8a-815d-456c-92c1-15178550402c"). InnerVolumeSpecName "kube-api-access-gd26n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:54:15.364578 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:15.364545    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd26n\" (UniqueName: \"kubernetes.io/projected/2d1b7d8a-815d-456c-92c1-15178550402c-kube-api-access-gd26n\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:54:15.973428 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:15.973394    2579 generic.go:356] "Generic (PLEG): container finished" podID="2d1b7d8a-815d-456c-92c1-15178550402c" containerID="17d1a85391f67a1deabc6a02bbe16c51de99e2d7b4c50d10e18fd2fa27234cf4" exitCode=137
Sep 19 10:54:15.973826 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:15.973432    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-9e0d93"
Sep 19 10:54:15.973826 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:15.973493    2579 scope.go:117] "RemoveContainer" containerID="17d1a85391f67a1deabc6a02bbe16c51de99e2d7b4c50d10e18fd2fa27234cf4"
Sep 19 10:54:15.982836 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:15.982821    2579 scope.go:117] "RemoveContainer" containerID="17d1a85391f67a1deabc6a02bbe16c51de99e2d7b4c50d10e18fd2fa27234cf4"
Sep 19 10:54:15.983098 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:15.983080    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17d1a85391f67a1deabc6a02bbe16c51de99e2d7b4c50d10e18fd2fa27234cf4\": container with ID starting with 17d1a85391f67a1deabc6a02bbe16c51de99e2d7b4c50d10e18fd2fa27234cf4 not found: ID does not exist" containerID="17d1a85391f67a1deabc6a02bbe16c51de99e2d7b4c50d10e18fd2fa27234cf4"
Sep 19 10:54:15.983167 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:15.983107    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17d1a85391f67a1deabc6a02bbe16c51de99e2d7b4c50d10e18fd2fa27234cf4"} err="failed to get container status \"17d1a85391f67a1deabc6a02bbe16c51de99e2d7b4c50d10e18fd2fa27234cf4\": rpc error: code = NotFound desc = could not find container \"17d1a85391f67a1deabc6a02bbe16c51de99e2d7b4c50d10e18fd2fa27234cf4\": container with ID starting with 17d1a85391f67a1deabc6a02bbe16c51de99e2d7b4c50d10e18fd2fa27234cf4 not found: ID does not exist"
Sep 19 10:54:15.984164 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:15.984142    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9074-9e0d93\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="2d1b7d8a-815d-456c-92c1-15178550402c" pod="opendatahub/test-curl-9074-9e0d93"
Sep 19 10:54:17.115956 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:17.115925    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d1b7d8a-815d-456c-92c1-15178550402c" path="/var/lib/kubelet/pods/2d1b7d8a-815d-456c-92c1-15178550402c/volumes"
Sep 19 10:54:17.143487 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:17.143465    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-98fe4e"
Sep 19 10:54:17.167649 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:17.167627    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:17.281901 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:17.281805    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4m2l\" (UniqueName: \"kubernetes.io/projected/abbb1fca-525e-44f5-94bd-58e30698300e-kube-api-access-w4m2l\") pod \"abbb1fca-525e-44f5-94bd-58e30698300e\" (UID: \"abbb1fca-525e-44f5-94bd-58e30698300e\") "
Sep 19 10:54:17.283979 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:17.283954    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abbb1fca-525e-44f5-94bd-58e30698300e-kube-api-access-w4m2l" (OuterVolumeSpecName: "kube-api-access-w4m2l") pod "abbb1fca-525e-44f5-94bd-58e30698300e" (UID: "abbb1fca-525e-44f5-94bd-58e30698300e"). InnerVolumeSpecName "kube-api-access-w4m2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:54:17.382773 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:17.382742    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4m2l\" (UniqueName: \"kubernetes.io/projected/abbb1fca-525e-44f5-94bd-58e30698300e-kube-api-access-w4m2l\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:54:17.490228 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:17.490200    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:17.585499 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:17.585429    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:17.987685 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:17.987653    2579 generic.go:356] "Generic (PLEG): container finished" podID="abbb1fca-525e-44f5-94bd-58e30698300e" containerID="e253616deb02d0e916812aa4226c05679155c7d39d99cf14bd22a6cd37b4a87f" exitCode=137
Sep 19 10:54:17.987841 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:17.987692    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9074-98fe4e"
Sep 19 10:54:17.987841 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:17.987749    2579 scope.go:117] "RemoveContainer" containerID="e253616deb02d0e916812aa4226c05679155c7d39d99cf14bd22a6cd37b4a87f"
Sep 19 10:54:17.997580 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:17.997564    2579 scope.go:117] "RemoveContainer" containerID="e253616deb02d0e916812aa4226c05679155c7d39d99cf14bd22a6cd37b4a87f"
Sep 19 10:54:17.997855 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:17.997836    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e253616deb02d0e916812aa4226c05679155c7d39d99cf14bd22a6cd37b4a87f\": container with ID starting with e253616deb02d0e916812aa4226c05679155c7d39d99cf14bd22a6cd37b4a87f not found: ID does not exist" containerID="e253616deb02d0e916812aa4226c05679155c7d39d99cf14bd22a6cd37b4a87f"
Sep 19 10:54:17.997914 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:17.997864    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e253616deb02d0e916812aa4226c05679155c7d39d99cf14bd22a6cd37b4a87f"} err="failed to get container status \"e253616deb02d0e916812aa4226c05679155c7d39d99cf14bd22a6cd37b4a87f\": rpc error: code = NotFound desc = could not find container \"e253616deb02d0e916812aa4226c05679155c7d39d99cf14bd22a6cd37b4a87f\": container with ID starting with e253616deb02d0e916812aa4226c05679155c7d39d99cf14bd22a6cd37b4a87f not found: ID does not exist"
Sep 19 10:54:17.998892 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:17.998868    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9074-98fe4e\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="abbb1fca-525e-44f5-94bd-58e30698300e" pod="opendatahub/test-curl-9074-98fe4e"
Sep 19 10:54:18.065583 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:18.065557    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:18.874659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:18.874578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:19.115982 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:19.115947    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abbb1fca-525e-44f5-94bd-58e30698300e" path="/var/lib/kubelet/pods/abbb1fca-525e-44f5-94bd-58e30698300e/volumes"
Sep 19 10:54:19.589677 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:19.589644    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:20.206266 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:20.206231    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:21.067636 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.067608    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:21.525859 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.525830    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"]
Sep 19 10:54:21.526205 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.526187    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abbb1fca-525e-44f5-94bd-58e30698300e" containerName="test-curl-9074-98fe4e"
Sep 19 10:54:21.526205 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.526198    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="abbb1fca-525e-44f5-94bd-58e30698300e" containerName="test-curl-9074-98fe4e"
Sep 19 10:54:21.526281 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.526268    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="abbb1fca-525e-44f5-94bd-58e30698300e" containerName="test-curl-9074-98fe4e"
Sep 19 10:54:21.530033 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.530014    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.533896 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.533872    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-72e262\"/\"default-dockercfg-gm55g\""
Sep 19 10:54:21.534018 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.534006    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-72e262\"/\"auth-test-model-72e262-kserve-self-signed-certs\""
Sep 19 10:54:21.534089 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.534010    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-72e262\"/\"openshift-service-ca.crt\""
Sep 19 10:54:21.534323 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.534292    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-72e262\"/\"kube-root-ca.crt\""
Sep 19 10:54:21.540261 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.539573    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"]
Sep 19 10:54:21.619577 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.619553    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q54xk\" (UniqueName: \"kubernetes.io/projected/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-kube-api-access-q54xk\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.619692 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.619595    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-dshm\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.619692 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.619623    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-tls-certs\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.619793 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.619683    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-model-cache\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.619793 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.619718    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-home\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.619793 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.619736    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-tmp-dir\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.720905 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.720880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-model-cache\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.721046 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.720913    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-home\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.721046 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.720930    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-tmp-dir\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.721046 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.720989    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q54xk\" (UniqueName: \"kubernetes.io/projected/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-kube-api-access-q54xk\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.721046 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.721017    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-dshm\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.721277 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.721148    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-tls-certs\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.721347 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.721323    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-home\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.721397 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.721345    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-tmp-dir\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.721397 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.721387    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-model-cache\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.723305 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.723278    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-dshm\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.723564 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.723546    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-tls-certs\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.738854 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.738830    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q54xk\" (UniqueName: \"kubernetes.io/projected/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-kube-api-access-q54xk\") pod \"auth-test-model-72e262-kserve-588f87d88c-nbfwb\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") " pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.744842 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.744818    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"]
Sep 19 10:54:21.748960 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.748940    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:21.752030 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.752008    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e6f44882-serving-cert\""
Sep 19 10:54:21.757711 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.757692    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"]
Sep 19 10:54:21.777263 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.777203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:21.821801 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.821777    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-maas-api-tls\") pod \"maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:21.821939 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.821813    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-metrics-certs\") pod \"maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:21.821991 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.821938    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m78wn\" (UniqueName: \"kubernetes.io/projected/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-kube-api-access-m78wn\") pod \"maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:21.821991 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.821977    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-openshift-service-ca\") pod \"maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:21.841682 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.841655    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:21.878680 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.878654    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:21.923125 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.923096    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m78wn\" (UniqueName: \"kubernetes.io/projected/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-kube-api-access-m78wn\") pod \"maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:21.923512 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.923145    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-openshift-service-ca\") pod \"maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:21.923512 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.923186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-maas-api-tls\") pod \"maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:21.923512 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.923208    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-metrics-certs\") pod \"maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:21.923512 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:21.923342    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-serving-cert: secret "maas-api-e2e-mt-e6f44882-serving-cert" not found
Sep 19 10:54:21.923512 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:54:21.923419    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-maas-api-tls podName:e1f64a55-97f8-4ef9-84f0-d22c97b66d9c nodeName:}" failed. No retries permitted until 2026-09-19 10:54:22.423398633 +0000 UTC m=+1183.335092149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-maas-api-tls") pod "maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl" (UID: "e1f64a55-97f8-4ef9-84f0-d22c97b66d9c") : secret "maas-api-e2e-mt-e6f44882-serving-cert" not found
Sep 19 10:54:21.924044 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.924023    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-openshift-service-ca\") pod \"maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:21.925744 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.925726    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-metrics-certs\") pod \"maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:21.931443 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.931425    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m78wn\" (UniqueName: \"kubernetes.io/projected/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-kube-api-access-m78wn\") pod \"maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:21.979655 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:21.979632    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:22.113824 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:22.113802    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"]
Sep 19 10:54:22.114985 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:54:22.114955    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8eeeee7_11ed_48e2_8b6f_3e2e319d721f.slice/crio-8e71e1fc163119908624ff03b81ad13789da7ca4aa6a314039fa1e627ffc07a3 WatchSource:0}: Error finding container 8e71e1fc163119908624ff03b81ad13789da7ca4aa6a314039fa1e627ffc07a3: Status 404 returned error can't find the container with id 8e71e1fc163119908624ff03b81ad13789da7ca4aa6a314039fa1e627ffc07a3
Sep 19 10:54:22.213258 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:22.213221    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:22.311067 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:22.311040    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:22.428504 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:22.428480    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-maas-api-tls\") pod \"maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:22.430830 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:22.430805    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-maas-api-tls\") pod \"maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:22.661028 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:22.660999    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:22.931414 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:22.931385    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"]
Sep 19 10:54:22.932793 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:54:22.932770    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f64a55_97f8_4ef9_84f0_d22c97b66d9c.slice/crio-3810fe9532e944a55118259a5ecf9ab9e61296b6cd7288c90d6bd88e3fa69c26 WatchSource:0}: Error finding container 3810fe9532e944a55118259a5ecf9ab9e61296b6cd7288c90d6bd88e3fa69c26: Status 404 returned error can't find the container with id 3810fe9532e944a55118259a5ecf9ab9e61296b6cd7288c90d6bd88e3fa69c26
Sep 19 10:54:23.016970 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:23.016901    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl" event={"ID":"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c","Type":"ContainerStarted","Data":"3810fe9532e944a55118259a5ecf9ab9e61296b6cd7288c90d6bd88e3fa69c26"}
Sep 19 10:54:23.018404 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:23.018378    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb" event={"ID":"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f","Type":"ContainerStarted","Data":"0e8c36f9be5b5089048b85f149854a52f37cfbfc38e56f039bc0943e6335c55e"}
Sep 19 10:54:23.018505 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:23.018417    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb" event={"ID":"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f","Type":"ContainerStarted","Data":"8e71e1fc163119908624ff03b81ad13789da7ca4aa6a314039fa1e627ffc07a3"}
Sep 19 10:54:23.034517 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:23.034472    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb" podStartSLOduration=2.034459328 podStartE2EDuration="2.034459328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:54:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:54:23.033305364 +0000 UTC m=+1183.944998885" watchObservedRunningTime="2026-09-19 10:54:23.034459328 +0000 UTC m=+1183.946152889"
Sep 19 10:54:24.025622 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:24.025583    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl" event={"ID":"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c","Type":"ContainerStarted","Data":"71fbd78b3956ff9992a959078b3545833e68b7a46ac342c535e3bd00cf3d4eb9"}
Sep 19 10:54:24.041762 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:24.041714    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl" podStartSLOduration=2.701815897 podStartE2EDuration="3.041699592s" totalImagesPullingTime="339.883695ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:54:24.040276416 +0000 UTC m=+1184.951969935" watchObservedRunningTime="2026-09-19 10:54:24.041699592 +0000 UTC m=+1184.953393111"
Sep 19 10:54:25.031162 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:25.031130    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:26.891595 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:26.891555    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:54:31.042098 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:31.042068    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:54:31.842162 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:31.842121    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:31.842378 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:31.842175    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:31.854900 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:31.854876    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:32.080066 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:32.080040    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:54:41.154051 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:41.154024    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:54:41.157311 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:41.157292    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:54:44.143322 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:44.143287    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:45.235450 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:45.235415    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:45.464301 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:45.464264    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:46.078470 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:46.078426    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:46.781457 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:46.781422    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:47.369619 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:47.369586    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:47.513672 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:47.513634    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:47.973174 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:47.973122    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:48.699086 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:48.699051    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:56.298172 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:56.298140    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:54:58.440582 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.440545    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"]
Sep 19 10:54:58.444265 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.444247    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.446846 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.446823    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-cc1b19\"/\"embedding-w0-cc1b19-kserve-self-signed-certs\""
Sep 19 10:54:58.453649 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.453629    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"]
Sep 19 10:54:58.533992 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.533967    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:54:58.567268 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.567244    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-dshm\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.567365 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.567290    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-model-cache\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.567365 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.567308    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-tmp-dir\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.567435 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.567367    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb57v\" (UniqueName: \"kubernetes.io/projected/3560ed30-4219-42ab-b59f-23e9d643ed49-kube-api-access-hb57v\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.567470 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.567436    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3560ed30-4219-42ab-b59f-23e9d643ed49-tls-certs\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.567506 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.567493    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-home\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.667991 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.667959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-model-cache\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.667991 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.667991    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-tmp-dir\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.668201 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.668012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb57v\" (UniqueName: \"kubernetes.io/projected/3560ed30-4219-42ab-b59f-23e9d643ed49-kube-api-access-hb57v\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.668201 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.668052    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3560ed30-4219-42ab-b59f-23e9d643ed49-tls-certs\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.668201 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.668075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-home\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.668201 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.668117    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-dshm\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.668457 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.668433    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-tmp-dir\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.668496 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.668443    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-model-cache\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.668556 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.668497    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-home\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.670352 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.670332    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-dshm\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.670536 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.670509    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3560ed30-4219-42ab-b59f-23e9d643ed49-tls-certs\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.675663 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.675644    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb57v\" (UniqueName: \"kubernetes.io/projected/3560ed30-4219-42ab-b59f-23e9d643ed49-kube-api-access-hb57v\") pod \"embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") " pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:58.753882 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:58.753825    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:54:59.024483 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:59.024456    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"]
Sep 19 10:54:59.025590 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:54:59.025565    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3560ed30_4219_42ab_b59f_23e9d643ed49.slice/crio-3106a99bb562162896be04981b3924616c6d700a107b717b4e9b344b623cc356 WatchSource:0}: Error finding container 3106a99bb562162896be04981b3924616c6d700a107b717b4e9b344b623cc356: Status 404 returned error can't find the container with id 3106a99bb562162896be04981b3924616c6d700a107b717b4e9b344b623cc356
Sep 19 10:54:59.227478 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:59.227440    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc" event={"ID":"3560ed30-4219-42ab-b59f-23e9d643ed49","Type":"ContainerStarted","Data":"1a0691736e4d54ad66dd198609d79967d9de01795179b4841d65d64c2d2c8147"}
Sep 19 10:54:59.227659 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:59.227486    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc" event={"ID":"3560ed30-4219-42ab-b59f-23e9d643ed49","Type":"ContainerStarted","Data":"3106a99bb562162896be04981b3924616c6d700a107b717b4e9b344b623cc356"}
Sep 19 10:54:59.244125 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:54:59.244078    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc" podStartSLOduration=1.244062646 podStartE2EDuration="1.244062646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:54:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:54:59.242392286 +0000 UTC m=+1220.154085808" watchObservedRunningTime="2026-09-19 10:54:59.244062646 +0000 UTC m=+1220.155756217"
Sep 19 10:55:01.917874 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:01.917838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:08.754402 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:08.754371    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:55:08.754809 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:08.754636    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:55:08.766776 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:08.766747    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:55:09.292095 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:09.292068    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:55:12.555381 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:12.555344    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:17.736865 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:17.736831    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:18.063237 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:18.063146    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:18.799985 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:18.799903    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:19.467998 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:19.467966    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:20.074277 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:20.074247    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:20.185692 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:20.185660    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:20.904520 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:20.904491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:21.768926 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:21.768897    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:22.371761 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:22.371730    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:22.722953 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:22.722924    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"]
Sep 19 10:55:22.749366 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:22.749337    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"]
Sep 19 10:55:22.792834 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:22.792806    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:23.158055 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:23.157975    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:23.758350 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:23.758320    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:23.898929 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:23.898896    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:55:33.448709 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:33.448677    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:33.511127 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:33.511095    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:33.612464 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:33.612435    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:33.712329 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:33.712249    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:33.961828 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:33.961796    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:34.068675 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:34.068582    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:34.687000 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:34.686934    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:35.445276 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:35.445246    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:35.582014 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:35.581989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:36.070979 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:36.070948    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:36.685224 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:36.685195    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:37.378485 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:37.378450    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:37.742207 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:37.742150    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn" podUID="70e95713-3c73-468c-ae4d-bf785a758838" containerName="main" containerID="cri-o://56f682689be34f6860d16fd8b5be01fdb26d2800306e90af6eb526661158c9ed" gracePeriod=45
Sep 19 10:55:37.768208 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:37.768179    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh" podUID="e9cea5d6-05e8-48ee-96ef-1808c82e953b" containerName="main" containerID="cri-o://d869a26af09b06b80fb011ba15d4aed5617ac9942c0e9f890cbe0848b1e83067" gracePeriod=45
Sep 19 10:55:38.028434 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.028406    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:55:38.031745 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.031717    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:55:38.130456 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130424    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-dshm\") pod \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") "
Sep 19 10:55:38.130653 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130472    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-home\") pod \"70e95713-3c73-468c-ae4d-bf785a758838\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") "
Sep 19 10:55:38.130653 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130539    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-model-cache\") pod \"70e95713-3c73-468c-ae4d-bf785a758838\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") "
Sep 19 10:55:38.130653 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130572    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tptxw\" (UniqueName: \"kubernetes.io/projected/70e95713-3c73-468c-ae4d-bf785a758838-kube-api-access-tptxw\") pod \"70e95713-3c73-468c-ae4d-bf785a758838\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") "
Sep 19 10:55:38.130653 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130601    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhhpn\" (UniqueName: \"kubernetes.io/projected/e9cea5d6-05e8-48ee-96ef-1808c82e953b-kube-api-access-lhhpn\") pod \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") "
Sep 19 10:55:38.130653 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130651    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-home\") pod \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") "
Sep 19 10:55:38.130939 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130681    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70e95713-3c73-468c-ae4d-bf785a758838-tls-certs\") pod \"70e95713-3c73-468c-ae4d-bf785a758838\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") "
Sep 19 10:55:38.130939 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130707    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-tmp-dir\") pod \"70e95713-3c73-468c-ae4d-bf785a758838\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") "
Sep 19 10:55:38.130939 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130736    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-dshm\") pod \"70e95713-3c73-468c-ae4d-bf785a758838\" (UID: \"70e95713-3c73-468c-ae4d-bf785a758838\") "
Sep 19 10:55:38.130939 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130779    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-tmp-dir\") pod \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") "
Sep 19 10:55:38.130939 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130802    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-model-cache\") pod \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") "
Sep 19 10:55:38.130939 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130800    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-model-cache" (OuterVolumeSpecName: "model-cache") pod "70e95713-3c73-468c-ae4d-bf785a758838" (UID: "70e95713-3c73-468c-ae4d-bf785a758838"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:55:38.130939 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130812    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-home" (OuterVolumeSpecName: "home") pod "70e95713-3c73-468c-ae4d-bf785a758838" (UID: "70e95713-3c73-468c-ae4d-bf785a758838"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:55:38.130939 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130839    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9cea5d6-05e8-48ee-96ef-1808c82e953b-tls-certs\") pod \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\" (UID: \"e9cea5d6-05e8-48ee-96ef-1808c82e953b\") "
Sep 19 10:55:38.130939 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.130901    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-home" (OuterVolumeSpecName: "home") pod "e9cea5d6-05e8-48ee-96ef-1808c82e953b" (UID: "e9cea5d6-05e8-48ee-96ef-1808c82e953b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:55:38.131423 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.131134    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e9cea5d6-05e8-48ee-96ef-1808c82e953b" (UID: "e9cea5d6-05e8-48ee-96ef-1808c82e953b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:55:38.131423 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.131297    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:55:38.131423 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.131318    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:55:38.131423 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.131331    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:55:38.131423 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.131344    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:55:38.131423 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.131356    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "70e95713-3c73-468c-ae4d-bf785a758838" (UID: "70e95713-3c73-468c-ae4d-bf785a758838"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:55:38.131423 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.131392    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-model-cache" (OuterVolumeSpecName: "model-cache") pod "e9cea5d6-05e8-48ee-96ef-1808c82e953b" (UID: "e9cea5d6-05e8-48ee-96ef-1808c82e953b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:55:38.133053 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.133029    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70e95713-3c73-468c-ae4d-bf785a758838-kube-api-access-tptxw" (OuterVolumeSpecName: "kube-api-access-tptxw") pod "70e95713-3c73-468c-ae4d-bf785a758838" (UID: "70e95713-3c73-468c-ae4d-bf785a758838"). InnerVolumeSpecName "kube-api-access-tptxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:55:38.133162 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.133061    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-dshm" (OuterVolumeSpecName: "dshm") pod "70e95713-3c73-468c-ae4d-bf785a758838" (UID: "70e95713-3c73-468c-ae4d-bf785a758838"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:55:38.133162 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.133150    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9cea5d6-05e8-48ee-96ef-1808c82e953b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e9cea5d6-05e8-48ee-96ef-1808c82e953b" (UID: "e9cea5d6-05e8-48ee-96ef-1808c82e953b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:55:38.133280 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.133261    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-dshm" (OuterVolumeSpecName: "dshm") pod "e9cea5d6-05e8-48ee-96ef-1808c82e953b" (UID: "e9cea5d6-05e8-48ee-96ef-1808c82e953b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:55:38.133334 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.133291    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9cea5d6-05e8-48ee-96ef-1808c82e953b-kube-api-access-lhhpn" (OuterVolumeSpecName: "kube-api-access-lhhpn") pod "e9cea5d6-05e8-48ee-96ef-1808c82e953b" (UID: "e9cea5d6-05e8-48ee-96ef-1808c82e953b"). InnerVolumeSpecName "kube-api-access-lhhpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:55:38.133578 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.133555    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70e95713-3c73-468c-ae4d-bf785a758838-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "70e95713-3c73-468c-ae4d-bf785a758838" (UID: "70e95713-3c73-468c-ae4d-bf785a758838"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:55:38.151494 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.151470    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:38.231960 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.231940    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70e95713-3c73-468c-ae4d-bf785a758838-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:55:38.231960 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.231959    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:55:38.232087 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.231968    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70e95713-3c73-468c-ae4d-bf785a758838-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:55:38.232087 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.231976    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:55:38.232087 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.231984    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9cea5d6-05e8-48ee-96ef-1808c82e953b-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:55:38.232087 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.231991    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9cea5d6-05e8-48ee-96ef-1808c82e953b-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:55:38.232087 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.231998    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tptxw\" (UniqueName: \"kubernetes.io/projected/70e95713-3c73-468c-ae4d-bf785a758838-kube-api-access-tptxw\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:55:38.232087 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.232007    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhhpn\" (UniqueName: \"kubernetes.io/projected/e9cea5d6-05e8-48ee-96ef-1808c82e953b-kube-api-access-lhhpn\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:55:38.458894 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.458856    2579 generic.go:356] "Generic (PLEG): container finished" podID="70e95713-3c73-468c-ae4d-bf785a758838" containerID="56f682689be34f6860d16fd8b5be01fdb26d2800306e90af6eb526661158c9ed" exitCode=0
Sep 19 10:55:38.459336 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.458903    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"
Sep 19 10:55:38.459336 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.458946    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn" event={"ID":"70e95713-3c73-468c-ae4d-bf785a758838","Type":"ContainerDied","Data":"56f682689be34f6860d16fd8b5be01fdb26d2800306e90af6eb526661158c9ed"}
Sep 19 10:55:38.459336 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.458991    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn" event={"ID":"70e95713-3c73-468c-ae4d-bf785a758838","Type":"ContainerDied","Data":"fd0f5243dece3cdfb9cec1db67602c2cff2238ac73101e07d158fd81f0cdd319"}
Sep 19 10:55:38.459336 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.459012    2579 scope.go:117] "RemoveContainer" containerID="56f682689be34f6860d16fd8b5be01fdb26d2800306e90af6eb526661158c9ed"
Sep 19 10:55:38.460546 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.460498    2579 generic.go:356] "Generic (PLEG): container finished" podID="e9cea5d6-05e8-48ee-96ef-1808c82e953b" containerID="d869a26af09b06b80fb011ba15d4aed5617ac9942c0e9f890cbe0848b1e83067" exitCode=0
Sep 19 10:55:38.460682 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.460549    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh" event={"ID":"e9cea5d6-05e8-48ee-96ef-1808c82e953b","Type":"ContainerDied","Data":"d869a26af09b06b80fb011ba15d4aed5617ac9942c0e9f890cbe0848b1e83067"}
Sep 19 10:55:38.460682 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.460564    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"
Sep 19 10:55:38.460682 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.460589    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh" event={"ID":"e9cea5d6-05e8-48ee-96ef-1808c82e953b","Type":"ContainerDied","Data":"63cd91e033a3a75db668d63f0ef4281325ec3c79876191bd075aaa4f06fde60e"}
Sep 19 10:55:38.469423 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.469399    2579 scope.go:117] "RemoveContainer" containerID="56f682689be34f6860d16fd8b5be01fdb26d2800306e90af6eb526661158c9ed"
Sep 19 10:55:38.469743 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:55:38.469723    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56f682689be34f6860d16fd8b5be01fdb26d2800306e90af6eb526661158c9ed\": container with ID starting with 56f682689be34f6860d16fd8b5be01fdb26d2800306e90af6eb526661158c9ed not found: ID does not exist" containerID="56f682689be34f6860d16fd8b5be01fdb26d2800306e90af6eb526661158c9ed"
Sep 19 10:55:38.469836 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.469754    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f682689be34f6860d16fd8b5be01fdb26d2800306e90af6eb526661158c9ed"} err="failed to get container status \"56f682689be34f6860d16fd8b5be01fdb26d2800306e90af6eb526661158c9ed\": rpc error: code = NotFound desc = could not find container \"56f682689be34f6860d16fd8b5be01fdb26d2800306e90af6eb526661158c9ed\": container with ID starting with 56f682689be34f6860d16fd8b5be01fdb26d2800306e90af6eb526661158c9ed not found: ID does not exist"
Sep 19 10:55:38.469836 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.469778    2579 scope.go:117] "RemoveContainer" containerID="d869a26af09b06b80fb011ba15d4aed5617ac9942c0e9f890cbe0848b1e83067"
Sep 19 10:55:38.478216 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.478201    2579 scope.go:117] "RemoveContainer" containerID="d869a26af09b06b80fb011ba15d4aed5617ac9942c0e9f890cbe0848b1e83067"
Sep 19 10:55:38.478459 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:55:38.478442    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d869a26af09b06b80fb011ba15d4aed5617ac9942c0e9f890cbe0848b1e83067\": container with ID starting with d869a26af09b06b80fb011ba15d4aed5617ac9942c0e9f890cbe0848b1e83067 not found: ID does not exist" containerID="d869a26af09b06b80fb011ba15d4aed5617ac9942c0e9f890cbe0848b1e83067"
Sep 19 10:55:38.478508 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.478465    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d869a26af09b06b80fb011ba15d4aed5617ac9942c0e9f890cbe0848b1e83067"} err="failed to get container status \"d869a26af09b06b80fb011ba15d4aed5617ac9942c0e9f890cbe0848b1e83067\": rpc error: code = NotFound desc = could not find container \"d869a26af09b06b80fb011ba15d4aed5617ac9942c0e9f890cbe0848b1e83067\": container with ID starting with d869a26af09b06b80fb011ba15d4aed5617ac9942c0e9f890cbe0848b1e83067 not found: ID does not exist"
Sep 19 10:55:38.484716 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.484695    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"]
Sep 19 10:55:38.488398 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.488377    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-580e01/distinct-2-w3-580e01-kserve-f99856f9d-hz8kn"]
Sep 19 10:55:38.498391 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.498372    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"]
Sep 19 10:55:38.504719 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:38.504699    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-580e01/simulator-w3-580e01-kserve-57886c6ff-926nh"]
Sep 19 10:55:39.116103 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:39.116066    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70e95713-3c73-468c-ae4d-bf785a758838" path="/var/lib/kubelet/pods/70e95713-3c73-468c-ae4d-bf785a758838/volumes"
Sep 19 10:55:39.116502 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:39.116488    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9cea5d6-05e8-48ee-96ef-1808c82e953b" path="/var/lib/kubelet/pods/e9cea5d6-05e8-48ee-96ef-1808c82e953b/volumes"
Sep 19 10:55:39.889966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:39.889932    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:39.990379 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:39.990349    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:40.142593 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:40.142501    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:42.322123 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:42.322085    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:42.419752 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:42.419691    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:43.393645 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:43.393609    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:55:51.593586 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:55:51.593553    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:56:05.671981 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:05.671940    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:14.311234 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:14.311198    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:14.966293 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:14.966258    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:15.565308 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:15.565278    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:16.159152 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:16.159125    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:16.865769 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:16.865732    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:17.461722 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:17.461687    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:18.058683 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:18.058655    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:19.392947 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:19.392912    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:56:26.580505 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:26.580469    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:26.942129 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:26.942105    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:27.747862 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:27.747835    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:28.335873 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:28.335839    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:28.830152 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:28.830114    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"]
Sep 19 10:56:28.830641 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:28.830383    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl" podUID="e1f64a55-97f8-4ef9-84f0-d22c97b66d9c" containerName="maas-api" containerID="cri-o://71fbd78b3956ff9992a959078b3545833e68b7a46ac342c535e3bd00cf3d4eb9" gracePeriod=30
Sep 19 10:56:28.855841 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:28.855818    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:28.927360 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:28.927318    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:28.985595 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:28.985565    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:29.041624 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.041592    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:29.078400 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.078378    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:56:29.150214 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.150147    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:29.194308 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.194284    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-maas-api-tls\") pod \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") "
Sep 19 10:56:29.194439 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.194371    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m78wn\" (UniqueName: \"kubernetes.io/projected/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-kube-api-access-m78wn\") pod \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") "
Sep 19 10:56:29.194492 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.194460    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-openshift-service-ca\") pod \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") "
Sep 19 10:56:29.194563 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.194510    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-metrics-certs\") pod \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\" (UID: \"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c\") "
Sep 19 10:56:29.194823 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.194795    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e1f64a55-97f8-4ef9-84f0-d22c97b66d9c" (UID: "e1f64a55-97f8-4ef9-84f0-d22c97b66d9c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:56:29.196478 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.196455    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e1f64a55-97f8-4ef9-84f0-d22c97b66d9c" (UID: "e1f64a55-97f8-4ef9-84f0-d22c97b66d9c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:56:29.196612 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.196487    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e1f64a55-97f8-4ef9-84f0-d22c97b66d9c" (UID: "e1f64a55-97f8-4ef9-84f0-d22c97b66d9c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:56:29.196612 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.196560    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-kube-api-access-m78wn" (OuterVolumeSpecName: "kube-api-access-m78wn") pod "e1f64a55-97f8-4ef9-84f0-d22c97b66d9c" (UID: "e1f64a55-97f8-4ef9-84f0-d22c97b66d9c"). InnerVolumeSpecName "kube-api-access-m78wn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:56:29.295381 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.295358    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-openshift-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:29.295381 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.295382    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-metrics-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:29.295554 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.295391    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-maas-api-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:29.295554 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.295399    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m78wn\" (UniqueName: \"kubernetes.io/projected/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c-kube-api-access-m78wn\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:29.734477 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.734441    2579 generic.go:356] "Generic (PLEG): container finished" podID="e1f64a55-97f8-4ef9-84f0-d22c97b66d9c" containerID="71fbd78b3956ff9992a959078b3545833e68b7a46ac342c535e3bd00cf3d4eb9" exitCode=0
Sep 19 10:56:29.734686 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.734483    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"
Sep 19 10:56:29.734686 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.734539    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl" event={"ID":"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c","Type":"ContainerDied","Data":"71fbd78b3956ff9992a959078b3545833e68b7a46ac342c535e3bd00cf3d4eb9"}
Sep 19 10:56:29.734686 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.734594    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl" event={"ID":"e1f64a55-97f8-4ef9-84f0-d22c97b66d9c","Type":"ContainerDied","Data":"3810fe9532e944a55118259a5ecf9ab9e61296b6cd7288c90d6bd88e3fa69c26"}
Sep 19 10:56:29.734686 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.734617    2579 scope.go:117] "RemoveContainer" containerID="71fbd78b3956ff9992a959078b3545833e68b7a46ac342c535e3bd00cf3d4eb9"
Sep 19 10:56:29.744434 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.744418    2579 scope.go:117] "RemoveContainer" containerID="71fbd78b3956ff9992a959078b3545833e68b7a46ac342c535e3bd00cf3d4eb9"
Sep 19 10:56:29.744712 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:56:29.744692    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71fbd78b3956ff9992a959078b3545833e68b7a46ac342c535e3bd00cf3d4eb9\": container with ID starting with 71fbd78b3956ff9992a959078b3545833e68b7a46ac342c535e3bd00cf3d4eb9 not found: ID does not exist" containerID="71fbd78b3956ff9992a959078b3545833e68b7a46ac342c535e3bd00cf3d4eb9"
Sep 19 10:56:29.744763 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.744720    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71fbd78b3956ff9992a959078b3545833e68b7a46ac342c535e3bd00cf3d4eb9"} err="failed to get container status \"71fbd78b3956ff9992a959078b3545833e68b7a46ac342c535e3bd00cf3d4eb9\": rpc error: code = NotFound desc = could not find container \"71fbd78b3956ff9992a959078b3545833e68b7a46ac342c535e3bd00cf3d4eb9\": container with ID starting with 71fbd78b3956ff9992a959078b3545833e68b7a46ac342c535e3bd00cf3d4eb9 not found: ID does not exist"
Sep 19 10:56:29.756098 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.756077    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"]
Sep 19 10:56:29.758580 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.758558    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6f44882-84b4d98d54-f2phl"]
Sep 19 10:56:29.886518 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:29.886489    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:30.669631 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:30.669592    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:31.115326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:31.115294    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1f64a55-97f8-4ef9-84f0-d22c97b66d9c" path="/var/lib/kubelet/pods/e1f64a55-97f8-4ef9-84f0-d22c97b66d9c/volumes"
Sep 19 10:56:31.297663 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:31.297629    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:31.572382 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:31.572349    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"]
Sep 19 10:56:31.577116 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:31.577091    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"]
Sep 19 10:56:31.584239 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:31.584112    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"]
Sep 19 10:56:31.592941 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:31.592919    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"]
Sep 19 10:56:31.606342 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:31.606319    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"]
Sep 19 10:56:31.657208 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:31.657183    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:31.965169 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:31.965138    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:32.071322 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:32.071287    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:32.675681 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:32.675653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:33.317111 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:33.317075    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:36.876969 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:36.876935    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:37.081389 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:37.081356    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:41.221006 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:41.220979    2579 scope.go:117] "RemoveContainer" containerID="6b8e3849efec88c091a74edb2fc4d22c1e41e9f702cebe6da013a221432953fc"
Sep 19 10:56:42.411638 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:42.411606    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"]
Sep 19 10:56:42.413198 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:42.411828    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4" podUID="5a80d11d-2e3c-4a2c-886b-341c8debe269" containerName="maas-api" containerID="cri-o://68e60b1d1ba997f9bb4af2b909975a4abd1d173c07c7cda4f204a1ede022ca31" gracePeriod=30
Sep 19 10:56:42.440827 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:42.440799    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:42.499268 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:42.499235    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:42.566478 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:42.566449    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:42.666383 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:42.666302    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:42.806257 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:42.806229    2579 generic.go:356] "Generic (PLEG): container finished" podID="5a80d11d-2e3c-4a2c-886b-341c8debe269" containerID="68e60b1d1ba997f9bb4af2b909975a4abd1d173c07c7cda4f204a1ede022ca31" exitCode=0
Sep 19 10:56:42.806423 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:42.806299    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4" event={"ID":"5a80d11d-2e3c-4a2c-886b-341c8debe269","Type":"ContainerDied","Data":"68e60b1d1ba997f9bb4af2b909975a4abd1d173c07c7cda4f204a1ede022ca31"}
Sep 19 10:56:42.857437 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:42.857414    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:56:42.972134 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:42.972101    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:43.022945 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.022921    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-metrics-certs\") pod \"5a80d11d-2e3c-4a2c-886b-341c8debe269\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") "
Sep 19 10:56:43.023055 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.022952    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-maas-api-tls\") pod \"5a80d11d-2e3c-4a2c-886b-341c8debe269\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") "
Sep 19 10:56:43.023055 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.023000    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqmjd\" (UniqueName: \"kubernetes.io/projected/5a80d11d-2e3c-4a2c-886b-341c8debe269-kube-api-access-pqmjd\") pod \"5a80d11d-2e3c-4a2c-886b-341c8debe269\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") "
Sep 19 10:56:43.023148 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.023058    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a80d11d-2e3c-4a2c-886b-341c8debe269-openshift-service-ca\") pod \"5a80d11d-2e3c-4a2c-886b-341c8debe269\" (UID: \"5a80d11d-2e3c-4a2c-886b-341c8debe269\") "
Sep 19 10:56:43.023439 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.023411    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a80d11d-2e3c-4a2c-886b-341c8debe269-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5a80d11d-2e3c-4a2c-886b-341c8debe269" (UID: "5a80d11d-2e3c-4a2c-886b-341c8debe269"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:56:43.024938 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.024916    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5a80d11d-2e3c-4a2c-886b-341c8debe269" (UID: "5a80d11d-2e3c-4a2c-886b-341c8debe269"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:56:43.025061 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.025047    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5a80d11d-2e3c-4a2c-886b-341c8debe269" (UID: "5a80d11d-2e3c-4a2c-886b-341c8debe269"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:56:43.025061 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.025049    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a80d11d-2e3c-4a2c-886b-341c8debe269-kube-api-access-pqmjd" (OuterVolumeSpecName: "kube-api-access-pqmjd") pod "5a80d11d-2e3c-4a2c-886b-341c8debe269" (UID: "5a80d11d-2e3c-4a2c-886b-341c8debe269"). InnerVolumeSpecName "kube-api-access-pqmjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:56:43.111576 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.111551    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:43.124168 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.124145    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-metrics-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:43.124168 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.124168    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a80d11d-2e3c-4a2c-886b-341c8debe269-maas-api-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:43.124321 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.124178    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqmjd\" (UniqueName: \"kubernetes.io/projected/5a80d11d-2e3c-4a2c-886b-341c8debe269-kube-api-access-pqmjd\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:43.124321 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.124187    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a80d11d-2e3c-4a2c-886b-341c8debe269-openshift-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:43.647071 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.647034    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:43.814094 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.814059    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4" event={"ID":"5a80d11d-2e3c-4a2c-886b-341c8debe269","Type":"ContainerDied","Data":"61dcd4d73b50b5ab2a15ab4d8a6a7b60f31f1e18af48ef55edae62fa86c1cd94"}
Sep 19 10:56:43.814094 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.814088    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"
Sep 19 10:56:43.814338 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.814115    2579 scope.go:117] "RemoveContainer" containerID="68e60b1d1ba997f9bb4af2b909975a4abd1d173c07c7cda4f204a1ede022ca31"
Sep 19 10:56:43.835658 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.835632    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"]
Sep 19 10:56:43.841342 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:43.841321    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-cc1b19-67b668f675-2qxn4"]
Sep 19 10:56:44.272483 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:44.272450    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:44.987756 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:44.987700    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:45.114992 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:45.114957    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a80d11d-2e3c-4a2c-886b-341c8debe269" path="/var/lib/kubelet/pods/5a80d11d-2e3c-4a2c-886b-341c8debe269/volumes"
Sep 19 10:56:45.682223 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:45.682190    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:46.597004 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.596960    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h" podUID="39c264fc-e8e7-4e35-979d-a0c035e67696" containerName="main" containerID="cri-o://c250180a6b935bb5c346ea401ceeb88b3b1671085fb62a264307a5100defadb9" gracePeriod=45
Sep 19 10:56:46.600698 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.600667    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m" podUID="cd9a7db1-d0be-4d01-ae33-fe2f087fcad3" containerName="main" containerID="cri-o://d66b9aefc7a53e5f3492c8f85468723d88e849abf4d676fbbb109ba3736d3c97" gracePeriod=45
Sep 19 10:56:46.611936 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.611906    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc" podUID="3560ed30-4219-42ab-b59f-23e9d643ed49" containerName="main" containerID="cri-o://1a0691736e4d54ad66dd198609d79967d9de01795179b4841d65d64c2d2c8147" gracePeriod=45
Sep 19 10:56:46.617928 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.617898    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4" podUID="f44ebb5c-d9ba-4898-947f-d1507a3986ff" containerName="main" containerID="cri-o://aaf31506e802760d9b30e4de999af482fda76a0ced85ac55d4a6331340c977e8" gracePeriod=45
Sep 19 10:56:46.624983 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.624957    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm" podUID="6f7f478d-bb4a-42fc-96a2-33992bef3d04" containerName="main" containerID="cri-o://50668378b44408f7f35d888c2ed08ab6ccd3a1332b086dd19bad1db6100d7379" gracePeriod=45
Sep 19 10:56:46.834498 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.834464    2579 generic.go:356] "Generic (PLEG): container finished" podID="3560ed30-4219-42ab-b59f-23e9d643ed49" containerID="1a0691736e4d54ad66dd198609d79967d9de01795179b4841d65d64c2d2c8147" exitCode=0
Sep 19 10:56:46.834666 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.834560    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc" event={"ID":"3560ed30-4219-42ab-b59f-23e9d643ed49","Type":"ContainerDied","Data":"1a0691736e4d54ad66dd198609d79967d9de01795179b4841d65d64c2d2c8147"}
Sep 19 10:56:46.836282 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.836261    2579 generic.go:356] "Generic (PLEG): container finished" podID="39c264fc-e8e7-4e35-979d-a0c035e67696" containerID="c250180a6b935bb5c346ea401ceeb88b3b1671085fb62a264307a5100defadb9" exitCode=0
Sep 19 10:56:46.836415 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.836371    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h" event={"ID":"39c264fc-e8e7-4e35-979d-a0c035e67696","Type":"ContainerDied","Data":"c250180a6b935bb5c346ea401ceeb88b3b1671085fb62a264307a5100defadb9"}
Sep 19 10:56:46.838259 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.838204    2579 generic.go:356] "Generic (PLEG): container finished" podID="cd9a7db1-d0be-4d01-ae33-fe2f087fcad3" containerID="d66b9aefc7a53e5f3492c8f85468723d88e849abf4d676fbbb109ba3736d3c97" exitCode=0
Sep 19 10:56:46.838358 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.838279    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m" event={"ID":"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3","Type":"ContainerDied","Data":"d66b9aefc7a53e5f3492c8f85468723d88e849abf4d676fbbb109ba3736d3c97"}
Sep 19 10:56:46.840360 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.840242    2579 generic.go:356] "Generic (PLEG): container finished" podID="6f7f478d-bb4a-42fc-96a2-33992bef3d04" containerID="50668378b44408f7f35d888c2ed08ab6ccd3a1332b086dd19bad1db6100d7379" exitCode=0
Sep 19 10:56:46.840360 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.840322    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm" event={"ID":"6f7f478d-bb4a-42fc-96a2-33992bef3d04","Type":"ContainerDied","Data":"50668378b44408f7f35d888c2ed08ab6ccd3a1332b086dd19bad1db6100d7379"}
Sep 19 10:56:46.842282 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.842239    2579 generic.go:356] "Generic (PLEG): container finished" podID="f44ebb5c-d9ba-4898-947f-d1507a3986ff" containerID="aaf31506e802760d9b30e4de999af482fda76a0ced85ac55d4a6331340c977e8" exitCode=0
Sep 19 10:56:46.842413 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.842299    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4" event={"ID":"f44ebb5c-d9ba-4898-947f-d1507a3986ff","Type":"ContainerDied","Data":"aaf31506e802760d9b30e4de999af482fda76a0ced85ac55d4a6331340c977e8"}
Sep 19 10:56:46.888584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.888562    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:56:46.933100 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.933071    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:56:46.977396 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.977373    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:56:46.983831 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.983808    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:56:46.987298 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:46.987277    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:56:47.056828 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.056802    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-dshm\") pod \"39c264fc-e8e7-4e35-979d-a0c035e67696\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") "
Sep 19 10:56:47.056990 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.056839    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3560ed30-4219-42ab-b59f-23e9d643ed49-tls-certs\") pod \"3560ed30-4219-42ab-b59f-23e9d643ed49\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") "
Sep 19 10:56:47.056990 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.056878    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-dshm\") pod \"3560ed30-4219-42ab-b59f-23e9d643ed49\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") "
Sep 19 10:56:47.056990 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.056946    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-home\") pod \"39c264fc-e8e7-4e35-979d-a0c035e67696\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") "
Sep 19 10:56:47.056990 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.056969    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-model-cache\") pod \"39c264fc-e8e7-4e35-979d-a0c035e67696\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") "
Sep 19 10:56:47.057213 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.057020    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-tmp-dir\") pod \"3560ed30-4219-42ab-b59f-23e9d643ed49\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") "
Sep 19 10:56:47.057273 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.057202    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-home" (OuterVolumeSpecName: "home") pod "39c264fc-e8e7-4e35-979d-a0c035e67696" (UID: "39c264fc-e8e7-4e35-979d-a0c035e67696"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.057273 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.057221    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-model-cache" (OuterVolumeSpecName: "model-cache") pod "39c264fc-e8e7-4e35-979d-a0c035e67696" (UID: "39c264fc-e8e7-4e35-979d-a0c035e67696"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.057400 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.057291    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39c264fc-e8e7-4e35-979d-a0c035e67696-tls-certs\") pod \"39c264fc-e8e7-4e35-979d-a0c035e67696\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") "
Sep 19 10:56:47.057400 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.057351    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hb57v\" (UniqueName: \"kubernetes.io/projected/3560ed30-4219-42ab-b59f-23e9d643ed49-kube-api-access-hb57v\") pod \"3560ed30-4219-42ab-b59f-23e9d643ed49\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") "
Sep 19 10:56:47.057400 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.057390    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-tmp-dir\") pod \"39c264fc-e8e7-4e35-979d-a0c035e67696\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") "
Sep 19 10:56:47.057582 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.057393    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3560ed30-4219-42ab-b59f-23e9d643ed49" (UID: "3560ed30-4219-42ab-b59f-23e9d643ed49"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.057582 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.057426    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x64dx\" (UniqueName: \"kubernetes.io/projected/39c264fc-e8e7-4e35-979d-a0c035e67696-kube-api-access-x64dx\") pod \"39c264fc-e8e7-4e35-979d-a0c035e67696\" (UID: \"39c264fc-e8e7-4e35-979d-a0c035e67696\") "
Sep 19 10:56:47.057582 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.057465    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-model-cache\") pod \"3560ed30-4219-42ab-b59f-23e9d643ed49\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") "
Sep 19 10:56:47.057582 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.057553    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-home\") pod \"3560ed30-4219-42ab-b59f-23e9d643ed49\" (UID: \"3560ed30-4219-42ab-b59f-23e9d643ed49\") "
Sep 19 10:56:47.058260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.058094    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "39c264fc-e8e7-4e35-979d-a0c035e67696" (UID: "39c264fc-e8e7-4e35-979d-a0c035e67696"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.058469 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.058433    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-model-cache" (OuterVolumeSpecName: "model-cache") pod "3560ed30-4219-42ab-b59f-23e9d643ed49" (UID: "3560ed30-4219-42ab-b59f-23e9d643ed49"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.058860 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.058797    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-home" (OuterVolumeSpecName: "home") pod "3560ed30-4219-42ab-b59f-23e9d643ed49" (UID: "3560ed30-4219-42ab-b59f-23e9d643ed49"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.059106 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.059089    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.059269 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.059249    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.059269 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.059269    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.059470 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.059281    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.059470 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.059293    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.059470 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.059304    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.059703 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.059565    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-dshm" (OuterVolumeSpecName: "dshm") pod "39c264fc-e8e7-4e35-979d-a0c035e67696" (UID: "39c264fc-e8e7-4e35-979d-a0c035e67696"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.059703 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.059581    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-dshm" (OuterVolumeSpecName: "dshm") pod "3560ed30-4219-42ab-b59f-23e9d643ed49" (UID: "3560ed30-4219-42ab-b59f-23e9d643ed49"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.059934 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.059910    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3560ed30-4219-42ab-b59f-23e9d643ed49-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3560ed30-4219-42ab-b59f-23e9d643ed49" (UID: "3560ed30-4219-42ab-b59f-23e9d643ed49"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:56:47.060055 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.060033    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3560ed30-4219-42ab-b59f-23e9d643ed49-kube-api-access-hb57v" (OuterVolumeSpecName: "kube-api-access-hb57v") pod "3560ed30-4219-42ab-b59f-23e9d643ed49" (UID: "3560ed30-4219-42ab-b59f-23e9d643ed49"). InnerVolumeSpecName "kube-api-access-hb57v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:56:47.060113 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.060095    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39c264fc-e8e7-4e35-979d-a0c035e67696-kube-api-access-x64dx" (OuterVolumeSpecName: "kube-api-access-x64dx") pod "39c264fc-e8e7-4e35-979d-a0c035e67696" (UID: "39c264fc-e8e7-4e35-979d-a0c035e67696"). InnerVolumeSpecName "kube-api-access-x64dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:56:47.060185 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.060171    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39c264fc-e8e7-4e35-979d-a0c035e67696-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "39c264fc-e8e7-4e35-979d-a0c035e67696" (UID: "39c264fc-e8e7-4e35-979d-a0c035e67696"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:56:47.159713 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.159638    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-model-cache\") pod \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") "
Sep 19 10:56:47.159713 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.159681    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-dshm\") pod \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") "
Sep 19 10:56:47.159713 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.159717    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7f478d-bb4a-42fc-96a2-33992bef3d04-tls-certs\") pod \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") "
Sep 19 10:56:47.159946 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.159744    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-home\") pod \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") "
Sep 19 10:56:47.159946 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.159790    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-home\") pod \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") "
Sep 19 10:56:47.159946 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.159821    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-tmp-dir\") pod \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") "
Sep 19 10:56:47.159946 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.159844    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-dshm\") pod \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") "
Sep 19 10:56:47.159946 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.159867    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-tmp-dir\") pod \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") "
Sep 19 10:56:47.159946 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.159902    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbztr\" (UniqueName: \"kubernetes.io/projected/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-kube-api-access-tbztr\") pod \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") "
Sep 19 10:56:47.159946 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.159925    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-model-cache" (OuterVolumeSpecName: "model-cache") pod "cd9a7db1-d0be-4d01-ae33-fe2f087fcad3" (UID: "cd9a7db1-d0be-4d01-ae33-fe2f087fcad3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.159946 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.159941    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f44ebb5c-d9ba-4898-947f-d1507a3986ff-tls-certs\") pod \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") "
Sep 19 10:56:47.160374 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.159972    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckf2x\" (UniqueName: \"kubernetes.io/projected/6f7f478d-bb4a-42fc-96a2-33992bef3d04-kube-api-access-ckf2x\") pod \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") "
Sep 19 10:56:47.160374 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160008    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-dshm\") pod \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") "
Sep 19 10:56:47.160374 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160034    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-home" (OuterVolumeSpecName: "home") pod "6f7f478d-bb4a-42fc-96a2-33992bef3d04" (UID: "6f7f478d-bb4a-42fc-96a2-33992bef3d04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.160374 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160044    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-home\") pod \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") "
Sep 19 10:56:47.160374 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160032    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-home" (OuterVolumeSpecName: "home") pod "cd9a7db1-d0be-4d01-ae33-fe2f087fcad3" (UID: "cd9a7db1-d0be-4d01-ae33-fe2f087fcad3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.160374 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160089    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnrl4\" (UniqueName: \"kubernetes.io/projected/f44ebb5c-d9ba-4898-947f-d1507a3986ff-kube-api-access-xnrl4\") pod \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") "
Sep 19 10:56:47.160374 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160120    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6f7f478d-bb4a-42fc-96a2-33992bef3d04" (UID: "6f7f478d-bb4a-42fc-96a2-33992bef3d04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.160374 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160136    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-tls-certs\") pod \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") "
Sep 19 10:56:47.160374 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160170    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-tmp-dir\") pod \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\" (UID: \"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3\") "
Sep 19 10:56:47.160374 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160200    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-model-cache\") pod \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\" (UID: \"6f7f478d-bb4a-42fc-96a2-33992bef3d04\") "
Sep 19 10:56:47.160374 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160228    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-model-cache\") pod \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\" (UID: \"f44ebb5c-d9ba-4898-947f-d1507a3986ff\") "
Sep 19 10:56:47.160966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160549    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.160966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160570    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x64dx\" (UniqueName: \"kubernetes.io/projected/39c264fc-e8e7-4e35-979d-a0c035e67696-kube-api-access-x64dx\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.160966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160585    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.160966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160601    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39c264fc-e8e7-4e35-979d-a0c035e67696-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.160966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160615    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3560ed30-4219-42ab-b59f-23e9d643ed49-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.160966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160627    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3560ed30-4219-42ab-b59f-23e9d643ed49-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.160966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160639    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.160966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160651    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39c264fc-e8e7-4e35-979d-a0c035e67696-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.160966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160663    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.160966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160676    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb57v\" (UniqueName: \"kubernetes.io/projected/3560ed30-4219-42ab-b59f-23e9d643ed49-kube-api-access-hb57v\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.160966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160595    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f44ebb5c-d9ba-4898-947f-d1507a3986ff" (UID: "f44ebb5c-d9ba-4898-947f-d1507a3986ff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.160966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160795    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-home" (OuterVolumeSpecName: "home") pod "f44ebb5c-d9ba-4898-947f-d1507a3986ff" (UID: "f44ebb5c-d9ba-4898-947f-d1507a3986ff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.160966 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.160893    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cd9a7db1-d0be-4d01-ae33-fe2f087fcad3" (UID: "cd9a7db1-d0be-4d01-ae33-fe2f087fcad3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.161664 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.161076    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-model-cache" (OuterVolumeSpecName: "model-cache") pod "6f7f478d-bb4a-42fc-96a2-33992bef3d04" (UID: "6f7f478d-bb4a-42fc-96a2-33992bef3d04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.161664 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.161241    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-model-cache" (OuterVolumeSpecName: "model-cache") pod "f44ebb5c-d9ba-4898-947f-d1507a3986ff" (UID: "f44ebb5c-d9ba-4898-947f-d1507a3986ff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.162313 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.162279    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-dshm" (OuterVolumeSpecName: "dshm") pod "6f7f478d-bb4a-42fc-96a2-33992bef3d04" (UID: "6f7f478d-bb4a-42fc-96a2-33992bef3d04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.162432 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.162380    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f7f478d-bb4a-42fc-96a2-33992bef3d04-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6f7f478d-bb4a-42fc-96a2-33992bef3d04" (UID: "6f7f478d-bb4a-42fc-96a2-33992bef3d04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:56:47.162500 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.162460    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-dshm" (OuterVolumeSpecName: "dshm") pod "cd9a7db1-d0be-4d01-ae33-fe2f087fcad3" (UID: "cd9a7db1-d0be-4d01-ae33-fe2f087fcad3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.162766 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.162744    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f44ebb5c-d9ba-4898-947f-d1507a3986ff-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f44ebb5c-d9ba-4898-947f-d1507a3986ff" (UID: "f44ebb5c-d9ba-4898-947f-d1507a3986ff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:56:47.163126 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.163103    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-kube-api-access-tbztr" (OuterVolumeSpecName: "kube-api-access-tbztr") pod "cd9a7db1-d0be-4d01-ae33-fe2f087fcad3" (UID: "cd9a7db1-d0be-4d01-ae33-fe2f087fcad3"). InnerVolumeSpecName "kube-api-access-tbztr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:56:47.163456 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.163430    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f44ebb5c-d9ba-4898-947f-d1507a3986ff-kube-api-access-xnrl4" (OuterVolumeSpecName: "kube-api-access-xnrl4") pod "f44ebb5c-d9ba-4898-947f-d1507a3986ff" (UID: "f44ebb5c-d9ba-4898-947f-d1507a3986ff"). InnerVolumeSpecName "kube-api-access-xnrl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:56:47.163753 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.163737    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-dshm" (OuterVolumeSpecName: "dshm") pod "f44ebb5c-d9ba-4898-947f-d1507a3986ff" (UID: "f44ebb5c-d9ba-4898-947f-d1507a3986ff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:56:47.163863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.163844    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd9a7db1-d0be-4d01-ae33-fe2f087fcad3" (UID: "cd9a7db1-d0be-4d01-ae33-fe2f087fcad3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:56:47.163903 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.163868    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7f478d-bb4a-42fc-96a2-33992bef3d04-kube-api-access-ckf2x" (OuterVolumeSpecName: "kube-api-access-ckf2x") pod "6f7f478d-bb4a-42fc-96a2-33992bef3d04" (UID: "6f7f478d-bb4a-42fc-96a2-33992bef3d04"). InnerVolumeSpecName "kube-api-access-ckf2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:56:47.261260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.261234    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f44ebb5c-d9ba-4898-947f-d1507a3986ff-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.261260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.261256    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckf2x\" (UniqueName: \"kubernetes.io/projected/6f7f478d-bb4a-42fc-96a2-33992bef3d04-kube-api-access-ckf2x\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.261260 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.261265    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.261591 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.261276    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.261591 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.261285    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnrl4\" (UniqueName: \"kubernetes.io/projected/f44ebb5c-d9ba-4898-947f-d1507a3986ff-kube-api-access-xnrl4\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.261591 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.261298    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.261591 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.261308    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.261591 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.261316    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.261591 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.261323    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.261591 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.261330    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f7f478d-bb4a-42fc-96a2-33992bef3d04-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.261591 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.261337    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f7f478d-bb4a-42fc-96a2-33992bef3d04-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.261591 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.261344    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44ebb5c-d9ba-4898-947f-d1507a3986ff-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.261591 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.261351    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.261591 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.261359    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbztr\" (UniqueName: \"kubernetes.io/projected/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3-kube-api-access-tbztr\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:56:47.849178 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.849144    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"
Sep 19 10:56:47.849686 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.849190    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc" event={"ID":"3560ed30-4219-42ab-b59f-23e9d643ed49","Type":"ContainerDied","Data":"3106a99bb562162896be04981b3924616c6d700a107b717b4e9b344b623cc356"}
Sep 19 10:56:47.849686 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.849240    2579 scope.go:117] "RemoveContainer" containerID="1a0691736e4d54ad66dd198609d79967d9de01795179b4841d65d64c2d2c8147"
Sep 19 10:56:47.850720 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.850700    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"
Sep 19 10:56:47.850836 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.850740    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h" event={"ID":"39c264fc-e8e7-4e35-979d-a0c035e67696","Type":"ContainerDied","Data":"44f28bdecda5f33fbde98f6eed3414d7ef2d773344261d2038a8cd7865e8ea6c"}
Sep 19 10:56:47.852276 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.852258    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"
Sep 19 10:56:47.852379 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.852270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m" event={"ID":"cd9a7db1-d0be-4d01-ae33-fe2f087fcad3","Type":"ContainerDied","Data":"d989c31413acdbf78f753bacfa7980f081502a13673d7426f38b77aa2fad8596"}
Sep 19 10:56:47.853982 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.853941    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"
Sep 19 10:56:47.853982 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.853957    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm" event={"ID":"6f7f478d-bb4a-42fc-96a2-33992bef3d04","Type":"ContainerDied","Data":"1f6bb7674a7fc270323e01a8d94116f6e1995f695d7ddc057d0d2a4f7336a3fc"}
Sep 19 10:56:47.855456 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.855433    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4" event={"ID":"f44ebb5c-d9ba-4898-947f-d1507a3986ff","Type":"ContainerDied","Data":"d0e4dc7a329eeddea6f511004a87d313fa5311391391e1fe1c8afc7bb800fb12"}
Sep 19 10:56:47.855557 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.855462    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"
Sep 19 10:56:47.860844 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.860816    2579 scope.go:117] "RemoveContainer" containerID="c250180a6b935bb5c346ea401ceeb88b3b1671085fb62a264307a5100defadb9"
Sep 19 10:56:47.868171 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.868154    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"]
Sep 19 10:56:47.870377 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.870362    2579 scope.go:117] "RemoveContainer" containerID="d66b9aefc7a53e5f3492c8f85468723d88e849abf4d676fbbb109ba3736d3c97"
Sep 19 10:56:47.870571 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.870550    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/embedding-w0-cc1b19-kserve-549bb5f6d9-8znjc"]
Sep 19 10:56:47.878104 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.878089    2579 scope.go:117] "RemoveContainer" containerID="50668378b44408f7f35d888c2ed08ab6ccd3a1332b086dd19bad1db6100d7379"
Sep 19 10:56:47.885700 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.885678    2579 scope.go:117] "RemoveContainer" containerID="aaf31506e802760d9b30e4de999af482fda76a0ced85ac55d4a6331340c977e8"
Sep 19 10:56:47.885771 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.885760    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"]
Sep 19 10:56:47.888119 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.888101    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/distinct-2-w0-cc1b19-kserve-788868db9f-ftd7h"]
Sep 19 10:56:47.897774 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.897757    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"]
Sep 19 10:56:47.900036 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.900018    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/premium-w0-cc1b19-kserve-56cd7c9c97-6fxb4"]
Sep 19 10:56:47.909182 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.909161    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"]
Sep 19 10:56:47.911001 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.910985    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/distinct-w0-cc1b19-kserve-5dccb47b8d-brd2m"]
Sep 19 10:56:47.920172 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.920154    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"]
Sep 19 10:56:47.924136 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:47.924118    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-cc1b19/simulator-w0-cc1b19-kserve-59857686d8-6jbmm"]
Sep 19 10:56:48.664076 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:48.664047    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:48.717864 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:48.717835    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:48.918210 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:48.918135    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:56:49.114981 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:49.114950    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3560ed30-4219-42ab-b59f-23e9d643ed49" path="/var/lib/kubelet/pods/3560ed30-4219-42ab-b59f-23e9d643ed49/volumes"
Sep 19 10:56:49.115346 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:49.115334    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39c264fc-e8e7-4e35-979d-a0c035e67696" path="/var/lib/kubelet/pods/39c264fc-e8e7-4e35-979d-a0c035e67696/volumes"
Sep 19 10:56:49.115710 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:49.115692    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f7f478d-bb4a-42fc-96a2-33992bef3d04" path="/var/lib/kubelet/pods/6f7f478d-bb4a-42fc-96a2-33992bef3d04/volumes"
Sep 19 10:56:49.116322 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:49.116307    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd9a7db1-d0be-4d01-ae33-fe2f087fcad3" path="/var/lib/kubelet/pods/cd9a7db1-d0be-4d01-ae33-fe2f087fcad3/volumes"
Sep 19 10:56:49.116708 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:49.116695    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f44ebb5c-d9ba-4898-947f-d1507a3986ff" path="/var/lib/kubelet/pods/f44ebb5c-d9ba-4898-947f-d1507a3986ff/volumes"
Sep 19 10:56:51.088130 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:51.088096    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:56:57.670085 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:56:57.670010    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:07.856136 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:07.856108    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:08.362730 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:08.362698    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:08.462781 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:08.462751    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:08.562852 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:08.562808    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:09.001542 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:09.001487    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:09.363881 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:09.363806    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:09.465178 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:09.465147    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:12.497452 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:12.497418    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:57:13.231864 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:13.231839    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:14.004032 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:14.004003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:14.512712 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:14.512684    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:15.918291 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:15.918259    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:19.496256 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:19.496224    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:19.595229 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:19.595202    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:19.736024 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:19.735998    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:20.939628 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:20.939595    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:21.056847 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:21.056816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:21.154541 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:21.154494    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:21.722269 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:21.722238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:21.809992 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:21.809963    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:21.909503 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:21.909472    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:23.092165 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:23.092129    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:28.881003 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:28.880974    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:28.980872 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:28.980842    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:29.524642 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:29.524609    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:34.191504 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:34.191460    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:57:35.252700 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:35.252668    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:35.355923 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:35.355890    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:35.452752 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:35.452722    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:45.917275 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:45.917244    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:46.167588 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:46.167485    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:57:50.481485 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:50.481447    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:57:58.648258 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:57:58.648221    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:05.589386 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:05.589350    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:58:07.010118 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:07.010088    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:09.583170 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:09.583127    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:12.775628 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:12.775595    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:12.827803 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:12.827777    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:13.099844 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:13.099761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:13.414141 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:13.414055    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:13.799570 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:13.799517    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:15.960117 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:15.960090    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:16.562777 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.562739    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:16.662753 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.662722    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:16.764941 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.764914    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:16.830725 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.830649    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9077-399cc3"]
Sep 19 10:58:16.831050 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831033    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3560ed30-4219-42ab-b59f-23e9d643ed49" containerName="main"
Sep 19 10:58:16.831115 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831054    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3560ed30-4219-42ab-b59f-23e9d643ed49" containerName="main"
Sep 19 10:58:16.831115 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831073    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a80d11d-2e3c-4a2c-886b-341c8debe269" containerName="maas-api"
Sep 19 10:58:16.831115 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831082    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a80d11d-2e3c-4a2c-886b-341c8debe269" containerName="maas-api"
Sep 19 10:58:16.831115 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831094    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f44ebb5c-d9ba-4898-947f-d1507a3986ff" containerName="main"
Sep 19 10:58:16.831115 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831103    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f44ebb5c-d9ba-4898-947f-d1507a3986ff" containerName="main"
Sep 19 10:58:16.831115 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831116    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd9a7db1-d0be-4d01-ae33-fe2f087fcad3" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831122    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd9a7db1-d0be-4d01-ae33-fe2f087fcad3" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831129    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9cea5d6-05e8-48ee-96ef-1808c82e953b" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831134    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9cea5d6-05e8-48ee-96ef-1808c82e953b" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831143    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70e95713-3c73-468c-ae4d-bf785a758838" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831149    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="70e95713-3c73-468c-ae4d-bf785a758838" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831159    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39c264fc-e8e7-4e35-979d-a0c035e67696" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831167    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="39c264fc-e8e7-4e35-979d-a0c035e67696" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831179    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f7f478d-bb4a-42fc-96a2-33992bef3d04" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831188    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f7f478d-bb4a-42fc-96a2-33992bef3d04" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831197    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1f64a55-97f8-4ef9-84f0-d22c97b66d9c" containerName="maas-api"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831202    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1f64a55-97f8-4ef9-84f0-d22c97b66d9c" containerName="maas-api"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831266    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="70e95713-3c73-468c-ae4d-bf785a758838" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831279    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f44ebb5c-d9ba-4898-947f-d1507a3986ff" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831284    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1f64a55-97f8-4ef9-84f0-d22c97b66d9c" containerName="maas-api"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831290    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f7f478d-bb4a-42fc-96a2-33992bef3d04" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831296    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="39c264fc-e8e7-4e35-979d-a0c035e67696" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831303    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a80d11d-2e3c-4a2c-886b-341c8debe269" containerName="maas-api"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831308    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9cea5d6-05e8-48ee-96ef-1808c82e953b" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831317    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd9a7db1-d0be-4d01-ae33-fe2f087fcad3" containerName="main"
Sep 19 10:58:16.831326 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.831326    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3560ed30-4219-42ab-b59f-23e9d643ed49" containerName="main"
Sep 19 10:58:16.834352 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.834337    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9077-399cc3"
Sep 19 10:58:16.838060 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.838036    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9d6w5\""
Sep 19 10:58:16.840693 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.840672    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9077-399cc3"]
Sep 19 10:58:16.919772 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:16.919742    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttdj4\" (UniqueName: \"kubernetes.io/projected/9b540116-00d6-45d4-ae4c-e11bfea9fc13-kube-api-access-ttdj4\") pod \"test-curl-9077-399cc3\" (UID: \"9b540116-00d6-45d4-ae4c-e11bfea9fc13\") " pod="opendatahub/test-curl-9077-399cc3"
Sep 19 10:58:17.020990 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:17.020962    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttdj4\" (UniqueName: \"kubernetes.io/projected/9b540116-00d6-45d4-ae4c-e11bfea9fc13-kube-api-access-ttdj4\") pod \"test-curl-9077-399cc3\" (UID: \"9b540116-00d6-45d4-ae4c-e11bfea9fc13\") " pod="opendatahub/test-curl-9077-399cc3"
Sep 19 10:58:17.029658 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:17.029636    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttdj4\" (UniqueName: \"kubernetes.io/projected/9b540116-00d6-45d4-ae4c-e11bfea9fc13-kube-api-access-ttdj4\") pod \"test-curl-9077-399cc3\" (UID: \"9b540116-00d6-45d4-ae4c-e11bfea9fc13\") " pod="opendatahub/test-curl-9077-399cc3"
Sep 19 10:58:17.144123 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:17.144065    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9077-399cc3"
Sep 19 10:58:17.410927 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:17.410900    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9077-399cc3"]
Sep 19 10:58:17.413516 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:58:17.413485    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b540116_00d6_45d4_ae4c_e11bfea9fc13.slice/crio-0dd357bd94397893fe662dedfe2f2cc72f8ed1d4bf3fe8744994c49d6809f8a0 WatchSource:0}: Error finding container 0dd357bd94397893fe662dedfe2f2cc72f8ed1d4bf3fe8744994c49d6809f8a0: Status 404 returned error can't find the container with id 0dd357bd94397893fe662dedfe2f2cc72f8ed1d4bf3fe8744994c49d6809f8a0
Sep 19 10:58:17.468284 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:17.468264    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:18.169330 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:18.169293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:18.317272 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:18.317244    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9077-399cc3" event={"ID":"9b540116-00d6-45d4-ae4c-e11bfea9fc13","Type":"ContainerStarted","Data":"932d6e3f1b1e1d2de603ea5143de47975788e3871c88af9cba07de9db1110ab4"}
Sep 19 10:58:18.317419 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:18.317279    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9077-399cc3" event={"ID":"9b540116-00d6-45d4-ae4c-e11bfea9fc13","Type":"ContainerStarted","Data":"0dd357bd94397893fe662dedfe2f2cc72f8ed1d4bf3fe8744994c49d6809f8a0"}
Sep 19 10:58:18.342297 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:18.342248    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-9077-399cc3" podStartSLOduration=2.342234097 podStartE2EDuration="2.342234097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:58:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:58:18.340835863 +0000 UTC m=+1419.252529384" watchObservedRunningTime="2026-09-19 10:58:18.342234097 +0000 UTC m=+1419.253927616"
Sep 19 10:58:18.903295 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:18.903220    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9077-399cc3"]
Sep 19 10:58:18.909256 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:18.909225    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:18.909712 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:18.909687    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9077-399cc3"]
Sep 19 10:58:19.321985 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:58:19.321957    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-9077-399cc3" container="test-curl-9077-399cc3"
Sep 19 10:58:19.556881 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.556847    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"]
Sep 19 10:58:19.557183 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.557171    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b540116-00d6-45d4-ae4c-e11bfea9fc13" containerName="test-curl-9077-399cc3"
Sep 19 10:58:19.557227 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.557184    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b540116-00d6-45d4-ae4c-e11bfea9fc13" containerName="test-curl-9077-399cc3"
Sep 19 10:58:19.557263 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.557248    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b540116-00d6-45d4-ae4c-e11bfea9fc13" containerName="test-curl-9077-399cc3"
Sep 19 10:58:19.560368 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.560351    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:19.562625 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.562603    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-45f425-serving-cert\""
Sep 19 10:58:19.569985 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.569962    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"]
Sep 19 10:58:19.640247 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.640132    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:19.740510 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.740466    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:19.745697 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.745675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8fccbd2-9c72-4b71-9ddf-6f166be23871-maas-api-tls\") pod \"maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:19.745777 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.745718    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8fccbd2-9c72-4b71-9ddf-6f166be23871-metrics-certs\") pod \"maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:19.745831 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.745788    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fccbd2-9c72-4b71-9ddf-6f166be23871-openshift-service-ca\") pod \"maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:19.745868 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.745831    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6gw7\" (UniqueName: \"kubernetes.io/projected/a8fccbd2-9c72-4b71-9ddf-6f166be23871-kube-api-access-s6gw7\") pod \"maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:19.789250 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.789214    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:19.846940 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.846913    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6gw7\" (UniqueName: \"kubernetes.io/projected/a8fccbd2-9c72-4b71-9ddf-6f166be23871-kube-api-access-s6gw7\") pod \"maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:19.847041 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.846957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8fccbd2-9c72-4b71-9ddf-6f166be23871-maas-api-tls\") pod \"maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:19.847160 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.847138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8fccbd2-9c72-4b71-9ddf-6f166be23871-metrics-certs\") pod \"maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:19.847246 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.847231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fccbd2-9c72-4b71-9ddf-6f166be23871-openshift-service-ca\") pod \"maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:19.847828 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.847810    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fccbd2-9c72-4b71-9ddf-6f166be23871-openshift-service-ca\") pod \"maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:19.849443 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.849420    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8fccbd2-9c72-4b71-9ddf-6f166be23871-maas-api-tls\") pod \"maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:19.849546 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.849480    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8fccbd2-9c72-4b71-9ddf-6f166be23871-metrics-certs\") pod \"maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:19.854931 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.854910    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6gw7\" (UniqueName: \"kubernetes.io/projected/a8fccbd2-9c72-4b71-9ddf-6f166be23871-kube-api-access-s6gw7\") pod \"maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:19.871706 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:19.871680    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:20.070094 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:20.070049    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:20.143993 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:20.143910    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"]
Sep 19 10:58:20.145584 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:20.145560    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:20.146389 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:58:20.146367    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8fccbd2_9c72_4b71_9ddf_6f166be23871.slice/crio-7a7b4fae5cb362e19fa1fbbdf8eb4474451f523a213b7325df195c34dc12ef6c WatchSource:0}: Error finding container 7a7b4fae5cb362e19fa1fbbdf8eb4474451f523a213b7325df195c34dc12ef6c: Status 404 returned error can't find the container with id 7a7b4fae5cb362e19fa1fbbdf8eb4474451f523a213b7325df195c34dc12ef6c
Sep 19 10:58:20.148180 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:20.148162    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 10:58:20.328620 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:20.328590    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5" event={"ID":"a8fccbd2-9c72-4b71-9ddf-6f166be23871","Type":"ContainerStarted","Data":"7a7b4fae5cb362e19fa1fbbdf8eb4474451f523a213b7325df195c34dc12ef6c"}
Sep 19 10:58:20.329011 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:20.328664    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-9077-399cc3" podUID="9b540116-00d6-45d4-ae4c-e11bfea9fc13" containerName="test-curl-9077-399cc3" containerID="cri-o://932d6e3f1b1e1d2de603ea5143de47975788e3871c88af9cba07de9db1110ab4" gracePeriod=2
Sep 19 10:58:20.794265 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:20.794240    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:58:21.335908 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:21.335869    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5" event={"ID":"a8fccbd2-9c72-4b71-9ddf-6f166be23871","Type":"ContainerStarted","Data":"6c5bd1344506fd31fc00ae64ce4cec3027ab89ad5d72041d09a1a057aa2fb125"}
Sep 19 10:58:21.336302 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:21.335995    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:21.353812 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:21.353769    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5" podStartSLOduration=1.795647712 podStartE2EDuration="2.353751286s" totalImagesPullingTime="558.103574ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:58:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:58:21.352375474 +0000 UTC m=+1422.264068993" watchObservedRunningTime="2026-09-19 10:58:21.353751286 +0000 UTC m=+1422.265444805"
Sep 19 10:58:21.727619 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:21.727588    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"]
Sep 19 10:58:21.744029 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:21.744006    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:22.513027 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:22.513000    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9077-399cc3"
Sep 19 10:58:22.675298 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:22.675275    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttdj4\" (UniqueName: \"kubernetes.io/projected/9b540116-00d6-45d4-ae4c-e11bfea9fc13-kube-api-access-ttdj4\") pod \"9b540116-00d6-45d4-ae4c-e11bfea9fc13\" (UID: \"9b540116-00d6-45d4-ae4c-e11bfea9fc13\") "
Sep 19 10:58:22.677229 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:22.677200    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b540116-00d6-45d4-ae4c-e11bfea9fc13-kube-api-access-ttdj4" (OuterVolumeSpecName: "kube-api-access-ttdj4") pod "9b540116-00d6-45d4-ae4c-e11bfea9fc13" (UID: "9b540116-00d6-45d4-ae4c-e11bfea9fc13"). InnerVolumeSpecName "kube-api-access-ttdj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:58:22.776275 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:22.776241    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttdj4\" (UniqueName: \"kubernetes.io/projected/9b540116-00d6-45d4-ae4c-e11bfea9fc13-kube-api-access-ttdj4\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:58:22.962016 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:22.961946    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:23.115797 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:23.115763    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b540116-00d6-45d4-ae4c-e11bfea9fc13" path="/var/lib/kubelet/pods/9b540116-00d6-45d4-ae4c-e11bfea9fc13/volumes"
Sep 19 10:58:23.347331 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:23.347243    2579 generic.go:356] "Generic (PLEG): container finished" podID="9b540116-00d6-45d4-ae4c-e11bfea9fc13" containerID="932d6e3f1b1e1d2de603ea5143de47975788e3871c88af9cba07de9db1110ab4" exitCode=137
Sep 19 10:58:23.347331 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:23.347283    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9077-399cc3"
Sep 19 10:58:23.347601 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:23.347344    2579 scope.go:117] "RemoveContainer" containerID="932d6e3f1b1e1d2de603ea5143de47975788e3871c88af9cba07de9db1110ab4"
Sep 19 10:58:23.356034 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:23.356013    2579 scope.go:117] "RemoveContainer" containerID="932d6e3f1b1e1d2de603ea5143de47975788e3871c88af9cba07de9db1110ab4"
Sep 19 10:58:23.356310 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:58:23.356285    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"932d6e3f1b1e1d2de603ea5143de47975788e3871c88af9cba07de9db1110ab4\": container with ID starting with 932d6e3f1b1e1d2de603ea5143de47975788e3871c88af9cba07de9db1110ab4 not found: ID does not exist" containerID="932d6e3f1b1e1d2de603ea5143de47975788e3871c88af9cba07de9db1110ab4"
Sep 19 10:58:23.356361 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:23.356319    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932d6e3f1b1e1d2de603ea5143de47975788e3871c88af9cba07de9db1110ab4"} err="failed to get container status \"932d6e3f1b1e1d2de603ea5143de47975788e3871c88af9cba07de9db1110ab4\": rpc error: code = NotFound desc = could not find container \"932d6e3f1b1e1d2de603ea5143de47975788e3871c88af9cba07de9db1110ab4\": container with ID starting with 932d6e3f1b1e1d2de603ea5143de47975788e3871c88af9cba07de9db1110ab4 not found: ID does not exist"
Sep 19 10:58:27.346558 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:27.346516    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:58:29.702954 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:29.702922    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:34.586070 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:34.586038    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:58:36.747033 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:36.746994    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb" podUID="c8eeeee7-11ed-48e2-8b6f-3e2e319d721f" containerName="main" containerID="cri-o://0e8c36f9be5b5089048b85f149854a52f37cfbfc38e56f039bc0943e6335c55e" gracePeriod=45
Sep 19 10:58:36.996274 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:36.996251    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:58:37.096585 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.096502    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-dshm\") pod \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") "
Sep 19 10:58:37.096752 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.096629    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-home\") pod \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") "
Sep 19 10:58:37.096752 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.096689    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-tls-certs\") pod \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") "
Sep 19 10:58:37.096752 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.096735    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-model-cache\") pod \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") "
Sep 19 10:58:37.096931 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.096761    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-tmp-dir\") pod \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") "
Sep 19 10:58:37.096931 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.096889    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-home" (OuterVolumeSpecName: "home") pod "c8eeeee7-11ed-48e2-8b6f-3e2e319d721f" (UID: "c8eeeee7-11ed-48e2-8b6f-3e2e319d721f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:58:37.096931 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.096905    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q54xk\" (UniqueName: \"kubernetes.io/projected/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-kube-api-access-q54xk\") pod \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\" (UID: \"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f\") "
Sep 19 10:58:37.097169 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.096986    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c8eeeee7-11ed-48e2-8b6f-3e2e319d721f" (UID: "c8eeeee7-11ed-48e2-8b6f-3e2e319d721f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:58:37.097169 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.097004    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-model-cache" (OuterVolumeSpecName: "model-cache") pod "c8eeeee7-11ed-48e2-8b6f-3e2e319d721f" (UID: "c8eeeee7-11ed-48e2-8b6f-3e2e319d721f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:58:37.097292 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.097205    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:58:37.097292 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.097220    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:58:37.097292 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.097228    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:58:37.098773 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.098747    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-dshm" (OuterVolumeSpecName: "dshm") pod "c8eeeee7-11ed-48e2-8b6f-3e2e319d721f" (UID: "c8eeeee7-11ed-48e2-8b6f-3e2e319d721f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 10:58:37.098955 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.098935    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c8eeeee7-11ed-48e2-8b6f-3e2e319d721f" (UID: "c8eeeee7-11ed-48e2-8b6f-3e2e319d721f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:58:37.099060 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.099042    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-kube-api-access-q54xk" (OuterVolumeSpecName: "kube-api-access-q54xk") pod "c8eeeee7-11ed-48e2-8b6f-3e2e319d721f" (UID: "c8eeeee7-11ed-48e2-8b6f-3e2e319d721f"). InnerVolumeSpecName "kube-api-access-q54xk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:58:37.197656 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.197621    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q54xk\" (UniqueName: \"kubernetes.io/projected/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-kube-api-access-q54xk\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:58:37.197656 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.197646    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:58:37.197656 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.197656    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:58:37.422895 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.422864    2579 generic.go:356] "Generic (PLEG): container finished" podID="c8eeeee7-11ed-48e2-8b6f-3e2e319d721f" containerID="0e8c36f9be5b5089048b85f149854a52f37cfbfc38e56f039bc0943e6335c55e" exitCode=0
Sep 19 10:58:37.423048 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.422913    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb" event={"ID":"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f","Type":"ContainerDied","Data":"0e8c36f9be5b5089048b85f149854a52f37cfbfc38e56f039bc0943e6335c55e"}
Sep 19 10:58:37.423048 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.422925    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"
Sep 19 10:58:37.423048 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.422946    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb" event={"ID":"c8eeeee7-11ed-48e2-8b6f-3e2e319d721f","Type":"ContainerDied","Data":"8e71e1fc163119908624ff03b81ad13789da7ca4aa6a314039fa1e627ffc07a3"}
Sep 19 10:58:37.423048 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.422968    2579 scope.go:117] "RemoveContainer" containerID="0e8c36f9be5b5089048b85f149854a52f37cfbfc38e56f039bc0943e6335c55e"
Sep 19 10:58:37.431551 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.431514    2579 scope.go:117] "RemoveContainer" containerID="0e8c36f9be5b5089048b85f149854a52f37cfbfc38e56f039bc0943e6335c55e"
Sep 19 10:58:37.431803 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:58:37.431781    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e8c36f9be5b5089048b85f149854a52f37cfbfc38e56f039bc0943e6335c55e\": container with ID starting with 0e8c36f9be5b5089048b85f149854a52f37cfbfc38e56f039bc0943e6335c55e not found: ID does not exist" containerID="0e8c36f9be5b5089048b85f149854a52f37cfbfc38e56f039bc0943e6335c55e"
Sep 19 10:58:37.431856 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.431810    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e8c36f9be5b5089048b85f149854a52f37cfbfc38e56f039bc0943e6335c55e"} err="failed to get container status \"0e8c36f9be5b5089048b85f149854a52f37cfbfc38e56f039bc0943e6335c55e\": rpc error: code = NotFound desc = could not find container \"0e8c36f9be5b5089048b85f149854a52f37cfbfc38e56f039bc0943e6335c55e\": container with ID starting with 0e8c36f9be5b5089048b85f149854a52f37cfbfc38e56f039bc0943e6335c55e not found: ID does not exist"
Sep 19 10:58:37.438834 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.438812    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"]
Sep 19 10:58:37.442336 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:37.442317    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/auth-test-model-72e262-kserve-588f87d88c-nbfwb"]
Sep 19 10:58:39.116079 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.116035    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8eeeee7-11ed-48e2-8b6f-3e2e319d721f" path="/var/lib/kubelet/pods/c8eeeee7-11ed-48e2-8b6f-3e2e319d721f/volumes"
Sep 19 10:58:39.865323 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.865285    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"]
Sep 19 10:58:39.865654 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.865640    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8eeeee7-11ed-48e2-8b6f-3e2e319d721f" containerName="main"
Sep 19 10:58:39.865715 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.865655    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8eeeee7-11ed-48e2-8b6f-3e2e319d721f" containerName="main"
Sep 19 10:58:39.865753 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.865718    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8eeeee7-11ed-48e2-8b6f-3e2e319d721f" containerName="main"
Sep 19 10:58:39.870087 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.870063    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:39.872764 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.872739    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-72e262\"/\"openshift-service-ca.crt\""
Sep 19 10:58:39.873951 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.873927    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-72e262\"/\"sub-test-model-72e262-kserve-self-signed-certs\""
Sep 19 10:58:39.874062 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.874009    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-72e262\"/\"default-dockercfg-gm55g\""
Sep 19 10:58:39.874062 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.873927    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-72e262\"/\"kube-root-ca.crt\""
Sep 19 10:58:39.878269 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.878248    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"]
Sep 19 10:58:39.890502 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.890482    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:39.921637 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.921608    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-home\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:39.921782 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.921642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-tmp-dir\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:39.921782 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.921671    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-dshm\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:39.921901 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.921787    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-model-cache\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:39.921901 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.921840    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-726zw\" (UniqueName: \"kubernetes.io/projected/e0f51f8a-7395-4902-b85e-4446a1d7ff70-kube-api-access-726zw\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:39.921984 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:39.921896    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0f51f8a-7395-4902-b85e-4446a1d7ff70-tls-certs\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:40.022594 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.022561    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-726zw\" (UniqueName: \"kubernetes.io/projected/e0f51f8a-7395-4902-b85e-4446a1d7ff70-kube-api-access-726zw\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:40.022725 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.022611    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0f51f8a-7395-4902-b85e-4446a1d7ff70-tls-certs\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:40.022725 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.022641    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-home\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:40.022725 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.022661    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-tmp-dir\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:40.022725 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.022688    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-dshm\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:40.022934 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.022744    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-model-cache\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:40.023056 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.023038    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-home\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:40.023127 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.023087    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-tmp-dir\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:40.023176 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.023140    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-model-cache\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:40.025019 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.024999    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-dshm\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:40.025302 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.025283    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0f51f8a-7395-4902-b85e-4446a1d7ff70-tls-certs\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:40.029860 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.029837    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-726zw\" (UniqueName: \"kubernetes.io/projected/e0f51f8a-7395-4902-b85e-4446a1d7ff70-kube-api-access-726zw\") pod \"sub-test-model-72e262-kserve-64bbb7d475-gjhk7\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") " pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:40.180336 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.180311    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:40.450554 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.450493    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"]
Sep 19 10:58:40.452245 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:58:40.452219    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0f51f8a_7395_4902_b85e_4446a1d7ff70.slice/crio-1a9622465e98dddcfa70fc6d7994b9be492f2f002a00dd103da1229e6188fde8 WatchSource:0}: Error finding container 1a9622465e98dddcfa70fc6d7994b9be492f2f002a00dd103da1229e6188fde8: Status 404 returned error can't find the container with id 1a9622465e98dddcfa70fc6d7994b9be492f2f002a00dd103da1229e6188fde8
Sep 19 10:58:40.972643 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:40.972611    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:41.448348 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:41.448314    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7" event={"ID":"e0f51f8a-7395-4902-b85e-4446a1d7ff70","Type":"ContainerStarted","Data":"564dc4e823cea43a4d6ba2e477a1c0ebfb0d29007e632bf66fb7f0c5da96425d"}
Sep 19 10:58:41.448727 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:41.448359    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7" event={"ID":"e0f51f8a-7395-4902-b85e-4446a1d7ff70","Type":"ContainerStarted","Data":"1a9622465e98dddcfa70fc6d7994b9be492f2f002a00dd103da1229e6188fde8"}
Sep 19 10:58:41.466670 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:41.466625    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7" podStartSLOduration=2.466606453 podStartE2EDuration="2.466606453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:58:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:58:41.463581397 +0000 UTC m=+1442.375274916" watchObservedRunningTime="2026-09-19 10:58:41.466606453 +0000 UTC m=+1442.378299973"
Sep 19 10:58:42.405987 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:42.405944    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:42.955620 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:42.955591    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:43.551064 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:43.551034    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:44.274377 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:44.274350    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:44.865751 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:44.865720    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:45.487587 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:45.487553    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:45.893767 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:45.893685    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:58:46.281676 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:46.281630    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:46.704460 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:46.704424    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:47.222465 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:47.222436    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:47.466386 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:47.466351    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:47.576388 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:47.576276    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:49.843424 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:49.843390    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:50.153760 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:50.153674    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:50.180733 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:50.180704    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:50.180863 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:50.180745    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:50.193648 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:50.193620    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:50.222935 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:50.222914    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:50.504988 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:50.504961    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 10:58:50.689367 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:50.689340    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:51.465318 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:51.465274    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:52.177273 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:52.177208    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:52.858144 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:52.858107    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:53.349050 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.349014    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"]
Sep 19 10:58:53.357338 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.357307    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:53.360219 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.360197    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a1c584-serving-cert\""
Sep 19 10:58:53.362318 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.362286    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"]
Sep 19 10:58:53.378931 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.378912    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:53.442436 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.442410    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-maas-api-tls\") pod \"maas-api-e2e-api-b-a1c584-d8755694f-7zqsr\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:53.442555 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.442443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-metrics-certs\") pod \"maas-api-e2e-api-b-a1c584-d8755694f-7zqsr\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:53.442555 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.442467    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5b9f\" (UniqueName: \"kubernetes.io/projected/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-kube-api-access-m5b9f\") pod \"maas-api-e2e-api-b-a1c584-d8755694f-7zqsr\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:53.442555 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.442544    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-openshift-service-ca\") pod \"maas-api-e2e-api-b-a1c584-d8755694f-7zqsr\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:53.479925 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.479900    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:53.543725 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.543699    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-openshift-service-ca\") pod \"maas-api-e2e-api-b-a1c584-d8755694f-7zqsr\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:53.543848 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.543771    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-maas-api-tls\") pod \"maas-api-e2e-api-b-a1c584-d8755694f-7zqsr\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:53.543848 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.543794    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-metrics-certs\") pod \"maas-api-e2e-api-b-a1c584-d8755694f-7zqsr\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:53.543848 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.543814    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5b9f\" (UniqueName: \"kubernetes.io/projected/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-kube-api-access-m5b9f\") pod \"maas-api-e2e-api-b-a1c584-d8755694f-7zqsr\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:53.544024 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:58:53.543919    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-serving-cert: secret "maas-api-e2e-api-b-a1c584-serving-cert" not found
Sep 19 10:58:53.544024 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:58:53.544001    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-maas-api-tls podName:bc8ce3ef-b4f4-48fe-9250-13fc53dc437a nodeName:}" failed. No retries permitted until 2026-09-19 10:58:54.04398124 +0000 UTC m=+1454.955674737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-maas-api-tls") pod "maas-api-e2e-api-b-a1c584-d8755694f-7zqsr" (UID: "bc8ce3ef-b4f4-48fe-9250-13fc53dc437a") : secret "maas-api-e2e-api-b-a1c584-serving-cert" not found
Sep 19 10:58:53.544390 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.544361    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-openshift-service-ca\") pod \"maas-api-e2e-api-b-a1c584-d8755694f-7zqsr\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:53.546412 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.546394    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-metrics-certs\") pod \"maas-api-e2e-api-b-a1c584-d8755694f-7zqsr\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:53.551921 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.551895    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5b9f\" (UniqueName: \"kubernetes.io/projected/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-kube-api-access-m5b9f\") pod \"maas-api-e2e-api-b-a1c584-d8755694f-7zqsr\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:53.852188 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.852159    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:53.956150 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:53.956120    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:54.029691 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:54.029661    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:58:54.049050 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:54.049025    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-maas-api-tls\") pod \"maas-api-e2e-api-b-a1c584-d8755694f-7zqsr\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:54.051444 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:54.051425    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-maas-api-tls\") pod \"maas-api-e2e-api-b-a1c584-d8755694f-7zqsr\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:54.271081 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:54.271052    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:54.540416 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:54.540389    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"]
Sep 19 10:58:54.541615 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:58:54.541587    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc8ce3ef_b4f4_48fe_9250_13fc53dc437a.slice/crio-f8ed2d732e5cfae7ad82dbdd14455a19cd1a72234edb723553f72298ca1a8ef9 WatchSource:0}: Error finding container f8ed2d732e5cfae7ad82dbdd14455a19cd1a72234edb723553f72298ca1a8ef9: Status 404 returned error can't find the container with id f8ed2d732e5cfae7ad82dbdd14455a19cd1a72234edb723553f72298ca1a8ef9
Sep 19 10:58:55.523245 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:55.523199    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr" event={"ID":"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a","Type":"ContainerStarted","Data":"af0aef7f0c2bd930061bd53d152c7807e347d7942d8d5352a3d4d24a0473041d"}
Sep 19 10:58:55.523628 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:55.523266    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:58:55.523628 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:55.523289    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr" event={"ID":"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a","Type":"ContainerStarted","Data":"f8ed2d732e5cfae7ad82dbdd14455a19cd1a72234edb723553f72298ca1a8ef9"}
Sep 19 10:58:55.541343 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:58:55.541299    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr" podStartSLOduration=2.2106269530000002 podStartE2EDuration="2.541283763s" totalImagesPullingTime="330.65681ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:58:55.539138317 +0000 UTC m=+1456.450831850" watchObservedRunningTime="2026-09-19 10:58:55.541283763 +0000 UTC m=+1456.452977324"
Sep 19 10:59:01.190885 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:01.190849    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:59:01.534339 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:01.534259    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 10:59:04.468996 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:04.468963    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:05.440925 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.440894    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"]
Sep 19 10:59:05.444196 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.444180    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.447993 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.447971    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-99b43e\"/\"sub-test-model-99b43e-kserve-self-signed-certs\""
Sep 19 10:59:05.448126 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.448004    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-99b43e\"/\"openshift-service-ca.crt\""
Sep 19 10:59:05.448126 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.448024    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-99b43e\"/\"kube-root-ca.crt\""
Sep 19 10:59:05.448126 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.448008    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-99b43e\"/\"default-dockercfg-q2qqw\""
Sep 19 10:59:05.451773 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.451749    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"]
Sep 19 10:59:05.457353 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.457331    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:05.549858 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.549829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-tmp-dir\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.550239 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.549901    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5de23d7d-7674-4bfb-8094-116c55fdb6bd-tls-certs\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.550239 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.549945    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-model-cache\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.550239 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.549979    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-dshm\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.550239 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.550004    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75srn\" (UniqueName: \"kubernetes.io/projected/5de23d7d-7674-4bfb-8094-116c55fdb6bd-kube-api-access-75srn\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.550239 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.550022    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-home\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.650783 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.650741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-tmp-dir\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.650971 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.650836    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5de23d7d-7674-4bfb-8094-116c55fdb6bd-tls-certs\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.650971 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.650866    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-model-cache\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.650971 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.650900    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-dshm\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.650971 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.650928    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75srn\" (UniqueName: \"kubernetes.io/projected/5de23d7d-7674-4bfb-8094-116c55fdb6bd-kube-api-access-75srn\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.650971 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.650957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-home\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.651243 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.651204    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-tmp-dir\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.651243 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.651228    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-model-cache\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.651361 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.651340    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-home\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.653173 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.653149    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-dshm\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.653492 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.653472    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5de23d7d-7674-4bfb-8094-116c55fdb6bd-tls-certs\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.658947 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.658926    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75srn\" (UniqueName: \"kubernetes.io/projected/5de23d7d-7674-4bfb-8094-116c55fdb6bd-kube-api-access-75srn\") pod \"sub-test-model-99b43e-kserve-77fdf78c-f8lrm\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.754835 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.754763    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:05.906274 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:05.906242    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:06.025518 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:06.025485    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"]
Sep 19 10:59:06.027501 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:59:06.027470    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5de23d7d_7674_4bfb_8094_116c55fdb6bd.slice/crio-479502fdcc874b2a469924c7855390179448d61b72d0aea8e12f636e477ee429 WatchSource:0}: Error finding container 479502fdcc874b2a469924c7855390179448d61b72d0aea8e12f636e477ee429: Status 404 returned error can't find the container with id 479502fdcc874b2a469924c7855390179448d61b72d0aea8e12f636e477ee429
Sep 19 10:59:06.589179 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:06.589142    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm" event={"ID":"5de23d7d-7674-4bfb-8094-116c55fdb6bd","Type":"ContainerStarted","Data":"dd791b6466ffeb28c0c38227836297bcf4b6312602d2fdc89c1845019785a560"}
Sep 19 10:59:06.589546 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:06.589189    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm" event={"ID":"5de23d7d-7674-4bfb-8094-116c55fdb6bd","Type":"ContainerStarted","Data":"479502fdcc874b2a469924c7855390179448d61b72d0aea8e12f636e477ee429"}
Sep 19 10:59:06.606248 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:06.606205    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm" podStartSLOduration=1.6061892599999998 podStartE2EDuration="1.60618926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:59:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:59:06.60479678 +0000 UTC m=+1467.516490299" watchObservedRunningTime="2026-09-19 10:59:06.60618926 +0000 UTC m=+1467.517882779"
Sep 19 10:59:14.858401 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:14.858366    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:15.755476 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:15.755444    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:15.755743 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:15.755720    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:15.767439 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:15.767414    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:15.914888 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:15.914850    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:16.407150 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:16.407121    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:16.650757 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:16.650729    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 10:59:17.185251 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:17.185212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:17.768457 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:17.768429    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:18.577682 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:18.577651    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:19.166912 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:19.166883    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:19.777273 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:19.777243    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:20.246799 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:20.246770    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:20.685942 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:20.685917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:20.786441 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:20.786392    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:24.047875 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:24.047838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:24.145679 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:24.145645    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:24.578136 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:24.578112    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:24.789992 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:24.789961    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:59:25.272411 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:25.272383    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:25.874445 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:25.874403    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:26.464885 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:26.464855    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:27.101695 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:27.101667    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:27.699846 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:27.699815    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:38.522280 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.522241    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"]
Sep 19 10:59:38.522754 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.522514    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5" podUID="a8fccbd2-9c72-4b71-9ddf-6f166be23871" containerName="maas-api" containerID="cri-o://6c5bd1344506fd31fc00ae64ce4cec3027ab89ad5d72041d09a1a057aa2fb125" gracePeriod=30
Sep 19 10:59:38.548925 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.548899    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:38.649367 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.649336    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:38.726310 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.726282    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:38.753709 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.753682    2579 generic.go:356] "Generic (PLEG): container finished" podID="a8fccbd2-9c72-4b71-9ddf-6f166be23871" containerID="6c5bd1344506fd31fc00ae64ce4cec3027ab89ad5d72041d09a1a057aa2fb125" exitCode=0
Sep 19 10:59:38.753834 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.753753    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5" event={"ID":"a8fccbd2-9c72-4b71-9ddf-6f166be23871","Type":"ContainerDied","Data":"6c5bd1344506fd31fc00ae64ce4cec3027ab89ad5d72041d09a1a057aa2fb125"}
Sep 19 10:59:38.769749 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.769727    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:59:38.776366 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.776312    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:38.840399 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.840375    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fccbd2-9c72-4b71-9ddf-6f166be23871-openshift-service-ca\") pod \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") "
Sep 19 10:59:38.840516 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.840433    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8fccbd2-9c72-4b71-9ddf-6f166be23871-metrics-certs\") pod \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") "
Sep 19 10:59:38.840516 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.840463    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8fccbd2-9c72-4b71-9ddf-6f166be23871-maas-api-tls\") pod \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") "
Sep 19 10:59:38.840516 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.840489    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6gw7\" (UniqueName: \"kubernetes.io/projected/a8fccbd2-9c72-4b71-9ddf-6f166be23871-kube-api-access-s6gw7\") pod \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\" (UID: \"a8fccbd2-9c72-4b71-9ddf-6f166be23871\") "
Sep 19 10:59:38.840757 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.840735    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8fccbd2-9c72-4b71-9ddf-6f166be23871-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a8fccbd2-9c72-4b71-9ddf-6f166be23871" (UID: "a8fccbd2-9c72-4b71-9ddf-6f166be23871"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 10:59:38.842610 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.842584    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8fccbd2-9c72-4b71-9ddf-6f166be23871-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a8fccbd2-9c72-4b71-9ddf-6f166be23871" (UID: "a8fccbd2-9c72-4b71-9ddf-6f166be23871"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:59:38.842704 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.842676    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8fccbd2-9c72-4b71-9ddf-6f166be23871-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a8fccbd2-9c72-4b71-9ddf-6f166be23871" (UID: "a8fccbd2-9c72-4b71-9ddf-6f166be23871"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 10:59:38.842749 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.842722    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8fccbd2-9c72-4b71-9ddf-6f166be23871-kube-api-access-s6gw7" (OuterVolumeSpecName: "kube-api-access-s6gw7") pod "a8fccbd2-9c72-4b71-9ddf-6f166be23871" (UID: "a8fccbd2-9c72-4b71-9ddf-6f166be23871"). InnerVolumeSpecName "kube-api-access-s6gw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:59:38.941629 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.941603    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8fccbd2-9c72-4b71-9ddf-6f166be23871-openshift-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:59:38.941629 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.941626    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8fccbd2-9c72-4b71-9ddf-6f166be23871-metrics-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:59:38.941629 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.941635    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8fccbd2-9c72-4b71-9ddf-6f166be23871-maas-api-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:59:38.941827 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.941643    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6gw7\" (UniqueName: \"kubernetes.io/projected/a8fccbd2-9c72-4b71-9ddf-6f166be23871-kube-api-access-s6gw7\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:59:38.998787 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:38.998730    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:59:39.076442 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:39.076359    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:39.187072 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:39.187031    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:39.760702 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:39.760671    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"
Sep 19 10:59:39.761122 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:39.760673    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5" event={"ID":"a8fccbd2-9c72-4b71-9ddf-6f166be23871","Type":"ContainerDied","Data":"7a7b4fae5cb362e19fa1fbbdf8eb4474451f523a213b7325df195c34dc12ef6c"}
Sep 19 10:59:39.761122 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:39.760802    2579 scope.go:117] "RemoveContainer" containerID="6c5bd1344506fd31fc00ae64ce4cec3027ab89ad5d72041d09a1a057aa2fb125"
Sep 19 10:59:39.777307 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:39.777284    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"]
Sep 19 10:59:39.779488 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:39.779464    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-45f425-6b675bbb46-l2pc5"]
Sep 19 10:59:40.050866 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:40.050785    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:40.679208 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:40.679175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:41.115938 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:41.115865    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8fccbd2-9c72-4b71-9ddf-6f166be23871" path="/var/lib/kubelet/pods/a8fccbd2-9c72-4b71-9ddf-6f166be23871/volumes"
Sep 19 10:59:41.196033 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:41.196010    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:59:41.200441 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:41.200423    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 10:59:41.438293 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:41.438256    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:42.069375 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:42.069329    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:42.669974 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:42.669935    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:44.940871 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:44.940822    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:44.997746 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:44.997715    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:45.193083 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:45.192999    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:46.510632 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:46.510603    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:46.611307 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:46.611276    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:47.257976 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:47.257948    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:47.860770 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:47.860735    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:48.415986 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:48.415954    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:49.110779 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:49.110742    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:49.916736 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:49.916700    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:50.244933 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:50.244904    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9077-07a680"]
Sep 19 10:59:50.245301 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:50.245254    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8fccbd2-9c72-4b71-9ddf-6f166be23871" containerName="maas-api"
Sep 19 10:59:50.245301 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:50.245266    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8fccbd2-9c72-4b71-9ddf-6f166be23871" containerName="maas-api"
Sep 19 10:59:50.245381 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:50.245327    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8fccbd2-9c72-4b71-9ddf-6f166be23871" containerName="maas-api"
Sep 19 10:59:50.248349 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:50.248331    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9077-07a680"
Sep 19 10:59:50.251178 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:50.251152    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9d6w5\""
Sep 19 10:59:50.254183 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:50.254161    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9077-07a680"]
Sep 19 10:59:50.337399 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:50.337372    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfw9g\" (UniqueName: \"kubernetes.io/projected/1453487a-f3b7-4fe6-8f8b-0ece4585e858-kube-api-access-bfw9g\") pod \"test-curl-9077-07a680\" (UID: \"1453487a-f3b7-4fe6-8f8b-0ece4585e858\") " pod="opendatahub/test-curl-9077-07a680"
Sep 19 10:59:50.437889 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:50.437858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfw9g\" (UniqueName: \"kubernetes.io/projected/1453487a-f3b7-4fe6-8f8b-0ece4585e858-kube-api-access-bfw9g\") pod \"test-curl-9077-07a680\" (UID: \"1453487a-f3b7-4fe6-8f8b-0ece4585e858\") " pod="opendatahub/test-curl-9077-07a680"
Sep 19 10:59:50.446401 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:50.446374    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfw9g\" (UniqueName: \"kubernetes.io/projected/1453487a-f3b7-4fe6-8f8b-0ece4585e858-kube-api-access-bfw9g\") pod \"test-curl-9077-07a680\" (UID: \"1453487a-f3b7-4fe6-8f8b-0ece4585e858\") " pod="opendatahub/test-curl-9077-07a680"
Sep 19 10:59:50.558751 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:50.558690    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9077-07a680"
Sep 19 10:59:50.826014 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:50.825990    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9077-07a680"]
Sep 19 10:59:50.827382 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:59:50.827358    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1453487a_f3b7_4fe6_8f8b_0ece4585e858.slice/crio-3906ae7899be458ad489544c6eb0d90373965b783caa059aba41ff831114a073 WatchSource:0}: Error finding container 3906ae7899be458ad489544c6eb0d90373965b783caa059aba41ff831114a073: Status 404 returned error can't find the container with id 3906ae7899be458ad489544c6eb0d90373965b783caa059aba41ff831114a073
Sep 19 10:59:51.824420 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:51.824381    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9077-07a680" event={"ID":"1453487a-f3b7-4fe6-8f8b-0ece4585e858","Type":"ContainerStarted","Data":"2eaed45318ea95db50fe879ede0cb7da9bf0b01d8814114f217094c971a6ace9"}
Sep 19 10:59:51.824835 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:51.824430    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9077-07a680" event={"ID":"1453487a-f3b7-4fe6-8f8b-0ece4585e858","Type":"ContainerStarted","Data":"3906ae7899be458ad489544c6eb0d90373965b783caa059aba41ff831114a073"}
Sep 19 10:59:51.837811 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:51.837757    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-9077-07a680" podStartSLOduration=1.837740425 podStartE2EDuration="1.837740425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:59:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:59:51.837221727 +0000 UTC m=+1512.748915245" watchObservedRunningTime="2026-09-19 10:59:51.837740425 +0000 UTC m=+1512.749433948"
Sep 19 10:59:52.394606 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:52.394577    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9077-07a680"]
Sep 19 10:59:52.398819 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:52.398791    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9077-07a680"]
Sep 19 10:59:52.513045 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:52.512973    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9077-2a1ec7"]
Sep 19 10:59:52.513314 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:52.513303    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1453487a-f3b7-4fe6-8f8b-0ece4585e858" containerName="test-curl-9077-07a680"
Sep 19 10:59:52.513354 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:52.513316    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1453487a-f3b7-4fe6-8f8b-0ece4585e858" containerName="test-curl-9077-07a680"
Sep 19 10:59:52.513397 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:52.513389    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1453487a-f3b7-4fe6-8f8b-0ece4585e858" containerName="test-curl-9077-07a680"
Sep 19 10:59:52.518191 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:52.517661    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9077-2a1ec7"
Sep 19 10:59:52.524397 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:52.524372    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9077-2a1ec7"]
Sep 19 10:59:52.658804 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:52.658775    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fztnk\" (UniqueName: \"kubernetes.io/projected/f3369be9-7a7e-4b64-82f6-9cf84ca4a781-kube-api-access-fztnk\") pod \"test-curl-9077-2a1ec7\" (UID: \"f3369be9-7a7e-4b64-82f6-9cf84ca4a781\") " pod="opendatahub/test-curl-9077-2a1ec7"
Sep 19 10:59:52.759612 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:52.759583    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fztnk\" (UniqueName: \"kubernetes.io/projected/f3369be9-7a7e-4b64-82f6-9cf84ca4a781-kube-api-access-fztnk\") pod \"test-curl-9077-2a1ec7\" (UID: \"f3369be9-7a7e-4b64-82f6-9cf84ca4a781\") " pod="opendatahub/test-curl-9077-2a1ec7"
Sep 19 10:59:52.766901 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:52.766847    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fztnk\" (UniqueName: \"kubernetes.io/projected/f3369be9-7a7e-4b64-82f6-9cf84ca4a781-kube-api-access-fztnk\") pod \"test-curl-9077-2a1ec7\" (UID: \"f3369be9-7a7e-4b64-82f6-9cf84ca4a781\") " pod="opendatahub/test-curl-9077-2a1ec7"
Sep 19 10:59:52.829448 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:59:52.829422    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-9077-07a680" container="test-curl-9077-07a680"
Sep 19 10:59:52.830344 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:52.830330    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9077-2a1ec7"
Sep 19 10:59:53.096687 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:53.096659    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9077-2a1ec7"]
Sep 19 10:59:53.097725 ip-10-0-140-105 kubenswrapper[2579]: W0919 10:59:53.097691    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3369be9_7a7e_4b64_82f6_9cf84ca4a781.slice/crio-71660420ebccf941c9c2f2019b07a1bd272e697df3c02fe98c6693250dcbad98 WatchSource:0}: Error finding container 71660420ebccf941c9c2f2019b07a1bd272e697df3c02fe98c6693250dcbad98: Status 404 returned error can't find the container with id 71660420ebccf941c9c2f2019b07a1bd272e697df3c02fe98c6693250dcbad98
Sep 19 10:59:53.836216 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:53.836174    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9077-2a1ec7" event={"ID":"f3369be9-7a7e-4b64-82f6-9cf84ca4a781","Type":"ContainerStarted","Data":"ef76b3fe75723d98b9b82e009c6d6b6adf13ae7a9cffaf0273e9a3165ec58eb5"}
Sep 19 10:59:53.836661 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:53.836229    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9077-2a1ec7" event={"ID":"f3369be9-7a7e-4b64-82f6-9cf84ca4a781","Type":"ContainerStarted","Data":"71660420ebccf941c9c2f2019b07a1bd272e697df3c02fe98c6693250dcbad98"}
Sep 19 10:59:53.836661 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:53.836250    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-9077-07a680" podUID="1453487a-f3b7-4fe6-8f8b-0ece4585e858" containerName="test-curl-9077-07a680" containerID="cri-o://2eaed45318ea95db50fe879ede0cb7da9bf0b01d8814114f217094c971a6ace9" gracePeriod=2
Sep 19 10:59:53.851892 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:53.851846    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-9077-2a1ec7" podStartSLOduration=1.8518312730000002 podStartE2EDuration="1.851831273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:59:53.849046592 +0000 UTC m=+1514.760740110" watchObservedRunningTime="2026-09-19 10:59:53.851831273 +0000 UTC m=+1514.763524821"
Sep 19 10:59:54.392457 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:54.392422    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9077-2a1ec7"]
Sep 19 10:59:54.394815 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:54.394789    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9077-2a1ec7"]
Sep 19 10:59:54.841199 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:59:54.841171    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-9077-2a1ec7" container="test-curl-9077-2a1ec7"
Sep 19 10:59:55.846634 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:55.846602    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-9077-2a1ec7" podUID="f3369be9-7a7e-4b64-82f6-9cf84ca4a781" containerName="test-curl-9077-2a1ec7" containerID="cri-o://ef76b3fe75723d98b9b82e009c6d6b6adf13ae7a9cffaf0273e9a3165ec58eb5" gracePeriod=2
Sep 19 10:59:56.004336 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:56.004313    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9077-07a680"
Sep 19 10:59:56.190832 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:56.190803    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfw9g\" (UniqueName: \"kubernetes.io/projected/1453487a-f3b7-4fe6-8f8b-0ece4585e858-kube-api-access-bfw9g\") pod \"1453487a-f3b7-4fe6-8f8b-0ece4585e858\" (UID: \"1453487a-f3b7-4fe6-8f8b-0ece4585e858\") "
Sep 19 10:59:56.192806 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:56.192785    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1453487a-f3b7-4fe6-8f8b-0ece4585e858-kube-api-access-bfw9g" (OuterVolumeSpecName: "kube-api-access-bfw9g") pod "1453487a-f3b7-4fe6-8f8b-0ece4585e858" (UID: "1453487a-f3b7-4fe6-8f8b-0ece4585e858"). InnerVolumeSpecName "kube-api-access-bfw9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:59:56.291502 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:56.291466    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfw9g\" (UniqueName: \"kubernetes.io/projected/1453487a-f3b7-4fe6-8f8b-0ece4585e858-kube-api-access-bfw9g\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:59:56.325301 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:56.325269    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"]
Sep 19 10:59:56.343927 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:56.343902    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:56.852737 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:56.852699    2579 generic.go:356] "Generic (PLEG): container finished" podID="1453487a-f3b7-4fe6-8f8b-0ece4585e858" containerID="2eaed45318ea95db50fe879ede0cb7da9bf0b01d8814114f217094c971a6ace9" exitCode=137
Sep 19 10:59:56.853208 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:56.852753    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9077-07a680"
Sep 19 10:59:56.853208 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:56.852789    2579 scope.go:117] "RemoveContainer" containerID="2eaed45318ea95db50fe879ede0cb7da9bf0b01d8814114f217094c971a6ace9"
Sep 19 10:59:56.862344 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:56.862327    2579 scope.go:117] "RemoveContainer" containerID="2eaed45318ea95db50fe879ede0cb7da9bf0b01d8814114f217094c971a6ace9"
Sep 19 10:59:56.862617 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:59:56.862599    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eaed45318ea95db50fe879ede0cb7da9bf0b01d8814114f217094c971a6ace9\": container with ID starting with 2eaed45318ea95db50fe879ede0cb7da9bf0b01d8814114f217094c971a6ace9 not found: ID does not exist" containerID="2eaed45318ea95db50fe879ede0cb7da9bf0b01d8814114f217094c971a6ace9"
Sep 19 10:59:56.862675 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:56.862625    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eaed45318ea95db50fe879ede0cb7da9bf0b01d8814114f217094c971a6ace9"} err="failed to get container status \"2eaed45318ea95db50fe879ede0cb7da9bf0b01d8814114f217094c971a6ace9\": rpc error: code = NotFound desc = could not find container \"2eaed45318ea95db50fe879ede0cb7da9bf0b01d8814114f217094c971a6ace9\": container with ID starting with 2eaed45318ea95db50fe879ede0cb7da9bf0b01d8814114f217094c971a6ace9 not found: ID does not exist"
Sep 19 10:59:56.863725 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:59:56.863693    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9077-07a680\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="1453487a-f3b7-4fe6-8f8b-0ece4585e858" pod="opendatahub/test-curl-9077-07a680"
Sep 19 10:59:57.115450 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:57.115373    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1453487a-f3b7-4fe6-8f8b-0ece4585e858" path="/var/lib/kubelet/pods/1453487a-f3b7-4fe6-8f8b-0ece4585e858/volumes"
Sep 19 10:59:57.223850 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:57.223820    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"]
Sep 19 10:59:57.240500 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:57.240468    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:58.028977 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:58.028951    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9077-2a1ec7"
Sep 19 10:59:58.208345 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:58.208315    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fztnk\" (UniqueName: \"kubernetes.io/projected/f3369be9-7a7e-4b64-82f6-9cf84ca4a781-kube-api-access-fztnk\") pod \"f3369be9-7a7e-4b64-82f6-9cf84ca4a781\" (UID: \"f3369be9-7a7e-4b64-82f6-9cf84ca4a781\") "
Sep 19 10:59:58.210312 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:58.210286    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3369be9-7a7e-4b64-82f6-9cf84ca4a781-kube-api-access-fztnk" (OuterVolumeSpecName: "kube-api-access-fztnk") pod "f3369be9-7a7e-4b64-82f6-9cf84ca4a781" (UID: "f3369be9-7a7e-4b64-82f6-9cf84ca4a781"). InnerVolumeSpecName "kube-api-access-fztnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 10:59:58.309375 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:58.309345    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fztnk\" (UniqueName: \"kubernetes.io/projected/f3369be9-7a7e-4b64-82f6-9cf84ca4a781-kube-api-access-fztnk\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 10:59:58.591199 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:58.591123    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 10:59:58.866095 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:58.866007    2579 generic.go:356] "Generic (PLEG): container finished" podID="f3369be9-7a7e-4b64-82f6-9cf84ca4a781" containerID="ef76b3fe75723d98b9b82e009c6d6b6adf13ae7a9cffaf0273e9a3165ec58eb5" exitCode=137
Sep 19 10:59:58.866095 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:58.866046    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9077-2a1ec7"
Sep 19 10:59:58.866322 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:58.866099    2579 scope.go:117] "RemoveContainer" containerID="ef76b3fe75723d98b9b82e009c6d6b6adf13ae7a9cffaf0273e9a3165ec58eb5"
Sep 19 10:59:58.874965 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:58.874951    2579 scope.go:117] "RemoveContainer" containerID="ef76b3fe75723d98b9b82e009c6d6b6adf13ae7a9cffaf0273e9a3165ec58eb5"
Sep 19 10:59:58.875220 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:59:58.875203    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef76b3fe75723d98b9b82e009c6d6b6adf13ae7a9cffaf0273e9a3165ec58eb5\": container with ID starting with ef76b3fe75723d98b9b82e009c6d6b6adf13ae7a9cffaf0273e9a3165ec58eb5 not found: ID does not exist" containerID="ef76b3fe75723d98b9b82e009c6d6b6adf13ae7a9cffaf0273e9a3165ec58eb5"
Sep 19 10:59:58.875283 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:58.875231    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef76b3fe75723d98b9b82e009c6d6b6adf13ae7a9cffaf0273e9a3165ec58eb5"} err="failed to get container status \"ef76b3fe75723d98b9b82e009c6d6b6adf13ae7a9cffaf0273e9a3165ec58eb5\": rpc error: code = NotFound desc = could not find container \"ef76b3fe75723d98b9b82e009c6d6b6adf13ae7a9cffaf0273e9a3165ec58eb5\": container with ID starting with ef76b3fe75723d98b9b82e009c6d6b6adf13ae7a9cffaf0273e9a3165ec58eb5 not found: ID does not exist"
Sep 19 10:59:58.876328 ip-10-0-140-105 kubenswrapper[2579]: E0919 10:59:58.876302    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-9077-2a1ec7\" is forbidden: User \"system:node:ip-10-0-140-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-105.ec2.internal' and this object" podUID="f3369be9-7a7e-4b64-82f6-9cf84ca4a781" pod="opendatahub/test-curl-9077-2a1ec7"
Sep 19 10:59:59.115428 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.115388    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3369be9-7a7e-4b64-82f6-9cf84ca4a781" path="/var/lib/kubelet/pods/f3369be9-7a7e-4b64-82f6-9cf84ca4a781/volumes"
Sep 19 10:59:59.745664 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.745631    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"]
Sep 19 10:59:59.746057 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.746039    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3369be9-7a7e-4b64-82f6-9cf84ca4a781" containerName="test-curl-9077-2a1ec7"
Sep 19 10:59:59.746152 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.746060    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3369be9-7a7e-4b64-82f6-9cf84ca4a781" containerName="test-curl-9077-2a1ec7"
Sep 19 10:59:59.746211 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.746154    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3369be9-7a7e-4b64-82f6-9cf84ca4a781" containerName="test-curl-9077-2a1ec7"
Sep 19 10:59:59.750409 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.750388    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 10:59:59.753072 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.753054    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-72e262\"/\"rate-test-model-72e262-kserve-self-signed-certs\""
Sep 19 10:59:59.757900 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.757882    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"]
Sep 19 10:59:59.769814 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.769794    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 10:59:59.924824 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.924793    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-dshm\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 10:59:59.924994 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.924836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-tmp-dir\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 10:59:59.924994 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.924857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-tls-certs\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 10:59:59.924994 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.924931    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-home\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 10:59:59.924994 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.924958    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-model-cache\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 10:59:59.925135 ip-10-0-140-105 kubenswrapper[2579]: I0919 10:59:59.925006    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4qdk\" (UniqueName: \"kubernetes.io/projected/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-kube-api-access-f4qdk\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:00.025982 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.025890    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-dshm\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:00.025982 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.025933    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-tmp-dir\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:00.026206 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.026049    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-tls-certs\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:00.026206 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.026117    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-home\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:00.026206 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.026155    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-model-cache\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:00.026206 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.026177    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qdk\" (UniqueName: \"kubernetes.io/projected/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-kube-api-access-f4qdk\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:00.026408 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.026278    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-tmp-dir\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:00.026453 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.026437    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-home\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:00.026508 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.026487    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-model-cache\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:00.028262 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.028243    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-dshm\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:00.028537 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.028507    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-tls-certs\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:00.033754 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.033733    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4qdk\" (UniqueName: \"kubernetes.io/projected/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-kube-api-access-f4qdk\") pod \"rate-test-model-72e262-kserve-5f9dd88bdc-zqwld\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") " pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:00.060681 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.060647    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:00.139297 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.139267    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-9ac6b3-29830260-cbl47"]
Sep 19 11:00:00.143931 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.143899    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-9ac6b3-29830260-cbl47"
Sep 19 11:00:00.146668 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.146648    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mckfb\""
Sep 19 11:00:00.149756 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.149700    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-9ac6b3-29830260-cbl47"]
Sep 19 11:00:00.156762 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.156742    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-9ac6b3-29830260-cbl47"
Sep 19 11:00:00.333792 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.333760    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"]
Sep 19 11:00:00.335039 ip-10-0-140-105 kubenswrapper[2579]: W0919 11:00:00.335009    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36fbe2ce_68cc_45cf_9cf0_94f272ae1f88.slice/crio-3fd5857de2140575747d4e3556701b5ea9209d19bf3873d9dc3c329df6b98d6d WatchSource:0}: Error finding container 3fd5857de2140575747d4e3556701b5ea9209d19bf3873d9dc3c329df6b98d6d: Status 404 returned error can't find the container with id 3fd5857de2140575747d4e3556701b5ea9209d19bf3873d9dc3c329df6b98d6d
Sep 19 11:00:00.427072 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.427041    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-9ac6b3-29830260-cbl47"]
Sep 19 11:00:00.428284 ip-10-0-140-105 kubenswrapper[2579]: W0919 11:00:00.428248    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod484200f3_e10b_4f0b_a1b7_47b8cb59a8c4.slice/crio-4b5c31a180e24340f8b5f8e9d10f3b16f237694a62f4de4da9d76c9d3afef994 WatchSource:0}: Error finding container 4b5c31a180e24340f8b5f8e9d10f3b16f237694a62f4de4da9d76c9d3afef994: Status 404 returned error can't find the container with id 4b5c31a180e24340f8b5f8e9d10f3b16f237694a62f4de4da9d76c9d3afef994
Sep 19 11:00:00.879050 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.878958    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld" event={"ID":"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88","Type":"ContainerStarted","Data":"3eeea7dcd091f771d812d02c74d4c9d010e5fcf97082d62cb54415bf6cf0604c"}
Sep 19 11:00:00.879050 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.879007    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld" event={"ID":"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88","Type":"ContainerStarted","Data":"3fd5857de2140575747d4e3556701b5ea9209d19bf3873d9dc3c329df6b98d6d"}
Sep 19 11:00:00.880584 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.880559    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-9ac6b3-29830260-cbl47" event={"ID":"484200f3-e10b-4f0b-a1b7-47b8cb59a8c4","Type":"ContainerStarted","Data":"642b2e05c7c8c0fec9bfa6fdd81064fe24c9cfcd544d556ce93d5fd5e52e2b59"}
Sep 19 11:00:00.880714 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.880601    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-9ac6b3-29830260-cbl47" event={"ID":"484200f3-e10b-4f0b-a1b7-47b8cb59a8c4","Type":"ContainerStarted","Data":"4b5c31a180e24340f8b5f8e9d10f3b16f237694a62f4de4da9d76c9d3afef994"}
Sep 19 11:00:00.894237 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.894191    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld" podStartSLOduration=1.894177548 podStartE2EDuration="1.894177548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:59:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:00:00.893545546 +0000 UTC m=+1521.805239060" watchObservedRunningTime="2026-09-19 11:00:00.894177548 +0000 UTC m=+1521.805871067"
Sep 19 11:00:00.907390 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:00.907347    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-9ac6b3-29830260-cbl47" podStartSLOduration=0.907329727 podStartE2EDuration="907.329727ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:00:00.90621564 +0000 UTC m=+1521.817909160" watchObservedRunningTime="2026-09-19 11:00:00.907329727 +0000 UTC m=+1521.819023250"
Sep 19 11:00:01.028998 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.028965    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"]
Sep 19 11:00:01.029264 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.029239    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr" podUID="bc8ce3ef-b4f4-48fe-9250-13fc53dc437a" containerName="maas-api" containerID="cri-o://af0aef7f0c2bd930061bd53d152c7807e347d7942d8d5352a3d4d24a0473041d" gracePeriod=30
Sep 19 11:00:01.059913 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.059890    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:01.124163 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.124125    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:01.176487 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.176453    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:01.243731 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.243696    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:01.294299 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.294268    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:01.326903 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.326881    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 11:00:01.337972 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.337952    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-maas-api-tls\") pod \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") "
Sep 19 11:00:01.338071 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.337991    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-metrics-certs\") pod \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") "
Sep 19 11:00:01.338071 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.338031    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5b9f\" (UniqueName: \"kubernetes.io/projected/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-kube-api-access-m5b9f\") pod \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") "
Sep 19 11:00:01.338071 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.338060    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-openshift-service-ca\") pod \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\" (UID: \"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a\") "
Sep 19 11:00:01.338468 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.338443    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bc8ce3ef-b4f4-48fe-9250-13fc53dc437a" (UID: "bc8ce3ef-b4f4-48fe-9250-13fc53dc437a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:00:01.340189 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.340168    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bc8ce3ef-b4f4-48fe-9250-13fc53dc437a" (UID: "bc8ce3ef-b4f4-48fe-9250-13fc53dc437a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:00:01.340284 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.340202    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bc8ce3ef-b4f4-48fe-9250-13fc53dc437a" (UID: "bc8ce3ef-b4f4-48fe-9250-13fc53dc437a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:00:01.340284 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.340219    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-kube-api-access-m5b9f" (OuterVolumeSpecName: "kube-api-access-m5b9f") pod "bc8ce3ef-b4f4-48fe-9250-13fc53dc437a" (UID: "bc8ce3ef-b4f4-48fe-9250-13fc53dc437a"). InnerVolumeSpecName "kube-api-access-m5b9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:00:01.438832 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.438794    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-maas-api-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:01.438832 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.438829    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-metrics-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:01.438832 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.438838    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5b9f\" (UniqueName: \"kubernetes.io/projected/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-kube-api-access-m5b9f\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:01.438832 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.438847    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a-openshift-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:01.672035 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.672003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:01.761491 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.761414    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:01.887348 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.887317    2579 generic.go:356] "Generic (PLEG): container finished" podID="484200f3-e10b-4f0b-a1b7-47b8cb59a8c4" containerID="642b2e05c7c8c0fec9bfa6fdd81064fe24c9cfcd544d556ce93d5fd5e52e2b59" exitCode=0
Sep 19 11:00:01.887552 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.887410    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-9ac6b3-29830260-cbl47" event={"ID":"484200f3-e10b-4f0b-a1b7-47b8cb59a8c4","Type":"ContainerDied","Data":"642b2e05c7c8c0fec9bfa6fdd81064fe24c9cfcd544d556ce93d5fd5e52e2b59"}
Sep 19 11:00:01.888775 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.888751    2579 generic.go:356] "Generic (PLEG): container finished" podID="bc8ce3ef-b4f4-48fe-9250-13fc53dc437a" containerID="af0aef7f0c2bd930061bd53d152c7807e347d7942d8d5352a3d4d24a0473041d" exitCode=0
Sep 19 11:00:01.888912 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.888794    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"
Sep 19 11:00:01.888912 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.888823    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr" event={"ID":"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a","Type":"ContainerDied","Data":"af0aef7f0c2bd930061bd53d152c7807e347d7942d8d5352a3d4d24a0473041d"}
Sep 19 11:00:01.888912 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.888857    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr" event={"ID":"bc8ce3ef-b4f4-48fe-9250-13fc53dc437a","Type":"ContainerDied","Data":"f8ed2d732e5cfae7ad82dbdd14455a19cd1a72234edb723553f72298ca1a8ef9"}
Sep 19 11:00:01.888912 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.888877    2579 scope.go:117] "RemoveContainer" containerID="af0aef7f0c2bd930061bd53d152c7807e347d7942d8d5352a3d4d24a0473041d"
Sep 19 11:00:01.898729 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.898711    2579 scope.go:117] "RemoveContainer" containerID="af0aef7f0c2bd930061bd53d152c7807e347d7942d8d5352a3d4d24a0473041d"
Sep 19 11:00:01.898981 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:00:01.898960    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af0aef7f0c2bd930061bd53d152c7807e347d7942d8d5352a3d4d24a0473041d\": container with ID starting with af0aef7f0c2bd930061bd53d152c7807e347d7942d8d5352a3d4d24a0473041d not found: ID does not exist" containerID="af0aef7f0c2bd930061bd53d152c7807e347d7942d8d5352a3d4d24a0473041d"
Sep 19 11:00:01.899028 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.898990    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af0aef7f0c2bd930061bd53d152c7807e347d7942d8d5352a3d4d24a0473041d"} err="failed to get container status \"af0aef7f0c2bd930061bd53d152c7807e347d7942d8d5352a3d4d24a0473041d\": rpc error: code = NotFound desc = could not find container \"af0aef7f0c2bd930061bd53d152c7807e347d7942d8d5352a3d4d24a0473041d\": container with ID starting with af0aef7f0c2bd930061bd53d152c7807e347d7942d8d5352a3d4d24a0473041d not found: ID does not exist"
Sep 19 11:00:01.915189 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.915158    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"]
Sep 19 11:00:01.919382 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:01.919360    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a1c584-d8755694f-7zqsr"]
Sep 19 11:00:02.353293 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:02.353259    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:03.024186 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:03.024161    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-9ac6b3-29830260-cbl47"
Sep 19 11:00:03.115046 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:03.115005    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc8ce3ef-b4f4-48fe-9250-13fc53dc437a" path="/var/lib/kubelet/pods/bc8ce3ef-b4f4-48fe-9250-13fc53dc437a/volumes"
Sep 19 11:00:03.334244 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:03.334210    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:03.778811 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:03.778776    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:03.901722 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:03.901689    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-9ac6b3-29830260-cbl47"
Sep 19 11:00:03.901890 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:03.901727    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-9ac6b3-29830260-cbl47" event={"ID":"484200f3-e10b-4f0b-a1b7-47b8cb59a8c4","Type":"ContainerDied","Data":"4b5c31a180e24340f8b5f8e9d10f3b16f237694a62f4de4da9d76c9d3afef994"}
Sep 19 11:00:03.901890 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:03.901765    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b5c31a180e24340f8b5f8e9d10f3b16f237694a62f4de4da9d76c9d3afef994"
Sep 19 11:00:04.379563 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:04.379494    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:07.305604 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:07.305569    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:07.365696 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:07.365633    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:07.627069 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:07.626983    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:08.166804 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:08.166769    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:08.870646 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:08.870611    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:09.578465 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:09.578433    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:10.061476 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:10.061447    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:10.061853 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:10.061486    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:10.074060 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:10.074035    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:10.176199 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:10.176163    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:10.882784 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:10.882749    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:10.949146 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:10.949117    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:00:11.344239 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.344202    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7" podUID="e0f51f8a-7395-4902-b85e-4446a1d7ff70" containerName="main" containerID="cri-o://564dc4e823cea43a4d6ba2e477a1c0ebfb0d29007e632bf66fb7f0c5da96425d" gracePeriod=45
Sep 19 11:00:11.568215 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.568177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:11.596359 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.596310    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 11:00:11.625998 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.625967    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-tmp-dir\") pod \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") "
Sep 19 11:00:11.626129 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.626079    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-dshm\") pod \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") "
Sep 19 11:00:11.626129 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.626118    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-home\") pod \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") "
Sep 19 11:00:11.626209 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.626191    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e0f51f8a-7395-4902-b85e-4446a1d7ff70" (UID: "e0f51f8a-7395-4902-b85e-4446a1d7ff70"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:00:11.626243 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.626202    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-726zw\" (UniqueName: \"kubernetes.io/projected/e0f51f8a-7395-4902-b85e-4446a1d7ff70-kube-api-access-726zw\") pod \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") "
Sep 19 11:00:11.626281 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.626267    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0f51f8a-7395-4902-b85e-4446a1d7ff70-tls-certs\") pod \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") "
Sep 19 11:00:11.626314 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.626303    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-model-cache\") pod \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\" (UID: \"e0f51f8a-7395-4902-b85e-4446a1d7ff70\") "
Sep 19 11:00:11.626805 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.626425    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-home" (OuterVolumeSpecName: "home") pod "e0f51f8a-7395-4902-b85e-4446a1d7ff70" (UID: "e0f51f8a-7395-4902-b85e-4446a1d7ff70"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:00:11.626955 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.626917    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-model-cache" (OuterVolumeSpecName: "model-cache") pod "e0f51f8a-7395-4902-b85e-4446a1d7ff70" (UID: "e0f51f8a-7395-4902-b85e-4446a1d7ff70"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:00:11.627416 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.627130    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:11.627641 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.627618    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:11.627737 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.627656    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:11.631729 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.630330    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-dshm" (OuterVolumeSpecName: "dshm") pod "e0f51f8a-7395-4902-b85e-4446a1d7ff70" (UID: "e0f51f8a-7395-4902-b85e-4446a1d7ff70"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:00:11.632011 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.631983    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0f51f8a-7395-4902-b85e-4446a1d7ff70-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e0f51f8a-7395-4902-b85e-4446a1d7ff70" (UID: "e0f51f8a-7395-4902-b85e-4446a1d7ff70"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:00:11.635785 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.635762    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0f51f8a-7395-4902-b85e-4446a1d7ff70-kube-api-access-726zw" (OuterVolumeSpecName: "kube-api-access-726zw") pod "e0f51f8a-7395-4902-b85e-4446a1d7ff70" (UID: "e0f51f8a-7395-4902-b85e-4446a1d7ff70"). InnerVolumeSpecName "kube-api-access-726zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:00:11.728356 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.728310    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-726zw\" (UniqueName: \"kubernetes.io/projected/e0f51f8a-7395-4902-b85e-4446a1d7ff70-kube-api-access-726zw\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:11.728356 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.728352    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0f51f8a-7395-4902-b85e-4446a1d7ff70-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:11.728356 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.728362    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0f51f8a-7395-4902-b85e-4446a1d7ff70-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:11.943482 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.943444    2579 generic.go:356] "Generic (PLEG): container finished" podID="e0f51f8a-7395-4902-b85e-4446a1d7ff70" containerID="564dc4e823cea43a4d6ba2e477a1c0ebfb0d29007e632bf66fb7f0c5da96425d" exitCode=0
Sep 19 11:00:11.943671 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.943506    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"
Sep 19 11:00:11.943671 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.943553    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7" event={"ID":"e0f51f8a-7395-4902-b85e-4446a1d7ff70","Type":"ContainerDied","Data":"564dc4e823cea43a4d6ba2e477a1c0ebfb0d29007e632bf66fb7f0c5da96425d"}
Sep 19 11:00:11.943671 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.943605    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7" event={"ID":"e0f51f8a-7395-4902-b85e-4446a1d7ff70","Type":"ContainerDied","Data":"1a9622465e98dddcfa70fc6d7994b9be492f2f002a00dd103da1229e6188fde8"}
Sep 19 11:00:11.943671 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.943634    2579 scope.go:117] "RemoveContainer" containerID="564dc4e823cea43a4d6ba2e477a1c0ebfb0d29007e632bf66fb7f0c5da96425d"
Sep 19 11:00:11.953196 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.953183    2579 scope.go:117] "RemoveContainer" containerID="564dc4e823cea43a4d6ba2e477a1c0ebfb0d29007e632bf66fb7f0c5da96425d"
Sep 19 11:00:11.953454 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:00:11.953433    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"564dc4e823cea43a4d6ba2e477a1c0ebfb0d29007e632bf66fb7f0c5da96425d\": container with ID starting with 564dc4e823cea43a4d6ba2e477a1c0ebfb0d29007e632bf66fb7f0c5da96425d not found: ID does not exist" containerID="564dc4e823cea43a4d6ba2e477a1c0ebfb0d29007e632bf66fb7f0c5da96425d"
Sep 19 11:00:11.953505 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.953461    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"564dc4e823cea43a4d6ba2e477a1c0ebfb0d29007e632bf66fb7f0c5da96425d"} err="failed to get container status \"564dc4e823cea43a4d6ba2e477a1c0ebfb0d29007e632bf66fb7f0c5da96425d\": rpc error: code = NotFound desc = could not find container \"564dc4e823cea43a4d6ba2e477a1c0ebfb0d29007e632bf66fb7f0c5da96425d\": container with ID starting with 564dc4e823cea43a4d6ba2e477a1c0ebfb0d29007e632bf66fb7f0c5da96425d not found: ID does not exist"
Sep 19 11:00:11.970927 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.970896    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"]
Sep 19 11:00:11.974340 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:11.974319    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/sub-test-model-72e262-kserve-64bbb7d475-gjhk7"]
Sep 19 11:00:12.243704 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.243599    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm" podUID="5de23d7d-7674-4bfb-8094-116c55fdb6bd" containerName="main" containerID="cri-o://dd791b6466ffeb28c0c38227836297bcf4b6312602d2fdc89c1845019785a560" gracePeriod=45
Sep 19 11:00:12.488839 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.488816    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 11:00:12.537087 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.537023    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5de23d7d-7674-4bfb-8094-116c55fdb6bd-tls-certs\") pod \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") "
Sep 19 11:00:12.537087 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.537071    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-tmp-dir\") pod \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") "
Sep 19 11:00:12.537274 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.537113    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-model-cache\") pod \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") "
Sep 19 11:00:12.537274 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.537131    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75srn\" (UniqueName: \"kubernetes.io/projected/5de23d7d-7674-4bfb-8094-116c55fdb6bd-kube-api-access-75srn\") pod \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") "
Sep 19 11:00:12.537274 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.537157    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-dshm\") pod \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") "
Sep 19 11:00:12.537274 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.537181    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-home\") pod \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\" (UID: \"5de23d7d-7674-4bfb-8094-116c55fdb6bd\") "
Sep 19 11:00:12.537472 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.537404    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5de23d7d-7674-4bfb-8094-116c55fdb6bd" (UID: "5de23d7d-7674-4bfb-8094-116c55fdb6bd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:00:12.537472 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.537421    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-model-cache" (OuterVolumeSpecName: "model-cache") pod "5de23d7d-7674-4bfb-8094-116c55fdb6bd" (UID: "5de23d7d-7674-4bfb-8094-116c55fdb6bd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:00:12.537609 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.537580    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-home" (OuterVolumeSpecName: "home") pod "5de23d7d-7674-4bfb-8094-116c55fdb6bd" (UID: "5de23d7d-7674-4bfb-8094-116c55fdb6bd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:00:12.539283 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.539253    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5de23d7d-7674-4bfb-8094-116c55fdb6bd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5de23d7d-7674-4bfb-8094-116c55fdb6bd" (UID: "5de23d7d-7674-4bfb-8094-116c55fdb6bd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:00:12.539356 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.539293    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5de23d7d-7674-4bfb-8094-116c55fdb6bd-kube-api-access-75srn" (OuterVolumeSpecName: "kube-api-access-75srn") pod "5de23d7d-7674-4bfb-8094-116c55fdb6bd" (UID: "5de23d7d-7674-4bfb-8094-116c55fdb6bd"). InnerVolumeSpecName "kube-api-access-75srn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:00:12.539356 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.539305    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-dshm" (OuterVolumeSpecName: "dshm") pod "5de23d7d-7674-4bfb-8094-116c55fdb6bd" (UID: "5de23d7d-7674-4bfb-8094-116c55fdb6bd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:00:12.637886 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.637858    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5de23d7d-7674-4bfb-8094-116c55fdb6bd-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:12.637886 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.637881    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:12.637886 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.637889    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:12.638077 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.637896    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75srn\" (UniqueName: \"kubernetes.io/projected/5de23d7d-7674-4bfb-8094-116c55fdb6bd-kube-api-access-75srn\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:12.638077 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.637908    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:12.638077 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.637915    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5de23d7d-7674-4bfb-8094-116c55fdb6bd-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:00:12.949585 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.949553    2579 generic.go:356] "Generic (PLEG): container finished" podID="5de23d7d-7674-4bfb-8094-116c55fdb6bd" containerID="dd791b6466ffeb28c0c38227836297bcf4b6312602d2fdc89c1845019785a560" exitCode=0
Sep 19 11:00:12.949775 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.949643    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm" event={"ID":"5de23d7d-7674-4bfb-8094-116c55fdb6bd","Type":"ContainerDied","Data":"dd791b6466ffeb28c0c38227836297bcf4b6312602d2fdc89c1845019785a560"}
Sep 19 11:00:12.949775 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.949659    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"
Sep 19 11:00:12.949775 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.949696    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm" event={"ID":"5de23d7d-7674-4bfb-8094-116c55fdb6bd","Type":"ContainerDied","Data":"479502fdcc874b2a469924c7855390179448d61b72d0aea8e12f636e477ee429"}
Sep 19 11:00:12.949775 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.949722    2579 scope.go:117] "RemoveContainer" containerID="dd791b6466ffeb28c0c38227836297bcf4b6312602d2fdc89c1845019785a560"
Sep 19 11:00:12.959102 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.959081    2579 scope.go:117] "RemoveContainer" containerID="dd791b6466ffeb28c0c38227836297bcf4b6312602d2fdc89c1845019785a560"
Sep 19 11:00:12.959362 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:00:12.959342    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd791b6466ffeb28c0c38227836297bcf4b6312602d2fdc89c1845019785a560\": container with ID starting with dd791b6466ffeb28c0c38227836297bcf4b6312602d2fdc89c1845019785a560 not found: ID does not exist" containerID="dd791b6466ffeb28c0c38227836297bcf4b6312602d2fdc89c1845019785a560"
Sep 19 11:00:12.959420 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.959369    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd791b6466ffeb28c0c38227836297bcf4b6312602d2fdc89c1845019785a560"} err="failed to get container status \"dd791b6466ffeb28c0c38227836297bcf4b6312602d2fdc89c1845019785a560\": rpc error: code = NotFound desc = could not find container \"dd791b6466ffeb28c0c38227836297bcf4b6312602d2fdc89c1845019785a560\": container with ID starting with dd791b6466ffeb28c0c38227836297bcf4b6312602d2fdc89c1845019785a560 not found: ID does not exist"
Sep 19 11:00:12.970661 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.970640    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"]
Sep 19 11:00:12.975505 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:12.975485    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99b43e/sub-test-model-99b43e-kserve-77fdf78c-f8lrm"]
Sep 19 11:00:13.115105 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:13.115076    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5de23d7d-7674-4bfb-8094-116c55fdb6bd" path="/var/lib/kubelet/pods/5de23d7d-7674-4bfb-8094-116c55fdb6bd/volumes"
Sep 19 11:00:13.115465 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:13.115453    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0f51f8a-7395-4902-b85e-4446a1d7ff70" path="/var/lib/kubelet/pods/e0f51f8a-7395-4902-b85e-4446a1d7ff70/volumes"
Sep 19 11:00:14.489492 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:14.489456    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:00:18.422257 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:18.422225    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-9ac6b3-29830260-cbl47"]
Sep 19 11:00:18.424264 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:18.424240    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-9ac6b3-29830260-cbl47"]
Sep 19 11:00:18.426136 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:18.426115    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:18.489514 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:18.489485    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:18.547406 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:18.547378    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:18.602686 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:18.602659    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:18.955947 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:18.955922    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:19.059209 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:19.059178    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:19.115200 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:19.115170    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="484200f3-e10b-4f0b-a1b7-47b8cb59a8c4" path="/var/lib/kubelet/pods/484200f3-e10b-4f0b-a1b7-47b8cb59a8c4/volumes"
Sep 19 11:00:19.589090 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:19.589059    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:20.233566 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:20.233513    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:20.956421 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:20.956386    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:24.251574 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.251539    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"]
Sep 19 11:00:24.252113 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.252092    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0f51f8a-7395-4902-b85e-4446a1d7ff70" containerName="main"
Sep 19 11:00:24.252162 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.252118    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0f51f8a-7395-4902-b85e-4446a1d7ff70" containerName="main"
Sep 19 11:00:24.252162 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.252130    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5de23d7d-7674-4bfb-8094-116c55fdb6bd" containerName="main"
Sep 19 11:00:24.252162 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.252139    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5de23d7d-7674-4bfb-8094-116c55fdb6bd" containerName="main"
Sep 19 11:00:24.252260 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.252172    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc8ce3ef-b4f4-48fe-9250-13fc53dc437a" containerName="maas-api"
Sep 19 11:00:24.252260 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.252180    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc8ce3ef-b4f4-48fe-9250-13fc53dc437a" containerName="maas-api"
Sep 19 11:00:24.252260 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.252191    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="484200f3-e10b-4f0b-a1b7-47b8cb59a8c4" containerName="cleanup"
Sep 19 11:00:24.252260 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.252199    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="484200f3-e10b-4f0b-a1b7-47b8cb59a8c4" containerName="cleanup"
Sep 19 11:00:24.252405 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.252308    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5de23d7d-7674-4bfb-8094-116c55fdb6bd" containerName="main"
Sep 19 11:00:24.252405 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.252320    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0f51f8a-7395-4902-b85e-4446a1d7ff70" containerName="main"
Sep 19 11:00:24.252405 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.252331    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="484200f3-e10b-4f0b-a1b7-47b8cb59a8c4" containerName="cleanup"
Sep 19 11:00:24.252405 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.252341    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc8ce3ef-b4f4-48fe-9250-13fc53dc437a" containerName="maas-api"
Sep 19 11:00:24.257253 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.257235    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.259686 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.259652    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-99b43e\"/\"rate-test-model-99b43e-kserve-self-signed-certs\""
Sep 19 11:00:24.260875 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.260831    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-99b43e\"/\"default-dockercfg-q2qqw\""
Sep 19 11:00:24.260991 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.260871    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-99b43e\"/\"openshift-service-ca.crt\""
Sep 19 11:00:24.261086 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.261066    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-99b43e\"/\"kube-root-ca.crt\""
Sep 19 11:00:24.261473 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.261430    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"]
Sep 19 11:00:24.277720 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.277694    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:24.332273 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.332246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-model-cache\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.332377 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.332281    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-home\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.332377 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.332312    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-tmp-dir\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.332501 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.332384    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvlmw\" (UniqueName: \"kubernetes.io/projected/2b3e5364-f781-4a27-aee9-b69fcefc556f-kube-api-access-fvlmw\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.332501 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.332431    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-dshm\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.332501 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.332476    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b3e5364-f781-4a27-aee9-b69fcefc556f-tls-certs\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.433150 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.433108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvlmw\" (UniqueName: \"kubernetes.io/projected/2b3e5364-f781-4a27-aee9-b69fcefc556f-kube-api-access-fvlmw\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.433150 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.433156    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-dshm\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.433361 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.433190    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b3e5364-f781-4a27-aee9-b69fcefc556f-tls-certs\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.433361 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.433299    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-model-cache\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.433361 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.433339    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-home\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.433361 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.433360    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-tmp-dir\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.433790 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.433761    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-tmp-dir\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.433790 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.433782    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-model-cache\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.433946 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.433833    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-home\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.435624 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.435600    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-dshm\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.435820 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.435806    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b3e5364-f781-4a27-aee9-b69fcefc556f-tls-certs\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.440058 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.440040    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvlmw\" (UniqueName: \"kubernetes.io/projected/2b3e5364-f781-4a27-aee9-b69fcefc556f-kube-api-access-fvlmw\") pod \"rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") " pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.569028 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.568953    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:24.842091 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.842064    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"]
Sep 19 11:00:24.843836 ip-10-0-140-105 kubenswrapper[2579]: W0919 11:00:24.843809    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b3e5364_f781_4a27_aee9_b69fcefc556f.slice/crio-41928698dd2bacb901a619be5b119d3428b6f806a20623d4f293627f91b180d4 WatchSource:0}: Error finding container 41928698dd2bacb901a619be5b119d3428b6f806a20623d4f293627f91b180d4: Status 404 returned error can't find the container with id 41928698dd2bacb901a619be5b119d3428b6f806a20623d4f293627f91b180d4
Sep 19 11:00:24.991463 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:24.991429    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:00:25.014417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:25.014390    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m" event={"ID":"2b3e5364-f781-4a27-aee9-b69fcefc556f","Type":"ContainerStarted","Data":"389ae259e797b5704a93fc3c096d4b04a607edb9756d8528a9239bb2d16ae2ad"}
Sep 19 11:00:25.014582 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:25.014434    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m" event={"ID":"2b3e5364-f781-4a27-aee9-b69fcefc556f","Type":"ContainerStarted","Data":"41928698dd2bacb901a619be5b119d3428b6f806a20623d4f293627f91b180d4"}
Sep 19 11:00:25.032044 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:25.031998    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m" podStartSLOduration=1.031984296 podStartE2EDuration="1.031984296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:00:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:00:25.03103611 +0000 UTC m=+1545.942729628" watchObservedRunningTime="2026-09-19 11:00:25.031984296 +0000 UTC m=+1545.943677838"
Sep 19 11:00:25.333416 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:25.333383    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:25.390543 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:25.390498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:25.580666 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:25.580639    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:26.074738 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:26.074707    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:26.810513 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:26.810467    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:27.464031 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:27.463998    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:28.055653 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:28.055623    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:28.667360 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:28.667323    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:34.569970 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:34.569932    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:34.570447 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:34.569992    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:34.582743 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:34.582717    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:35.080632 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:35.080602    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:00:36.285706 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:36.285662    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:00:36.336970 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:36.336943    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:36.464155 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:36.464125    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:36.530640 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:36.530615    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:36.596085 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:36.596000    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:36.653839 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:36.653796    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:36.958703 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:36.958669    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:37.059849 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:37.059814    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:37.507507 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:37.507469    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:38.280687 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:38.280653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:42.490811 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:42.490780    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:42.554697 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:42.554666    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:42.743237 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:42.743155    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:00:46.388195 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:00:46.388164    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:01:03.085963 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:03.085925    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:01:07.222594 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:07.222561    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"]
Sep 19 11:01:07.241444 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:07.241420    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:08.246414 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:08.246378    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"]
Sep 19 11:01:08.262375 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:08.262349    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:08.697346 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:08.697316    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:08.985905 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:08.985832    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:09.182762 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:09.182724    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:10.933472 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:10.933437    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:10.998890 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:10.998859    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:01:11.439604 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:11.439575    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:11.667442 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:11.667413    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:11.765875 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:11.765787    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:12.359659 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:12.359631    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:13.014339 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:13.014307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:13.073016 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:13.072986    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:13.210899 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:13.210869    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:13.466002 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:13.465975    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:13.565189 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:13.565158    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:22.242413 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.242372    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld" podUID="36fbe2ce-68cc-45cf-9cf0-94f272ae1f88" containerName="main" containerID="cri-o://3eeea7dcd091f771d812d02c74d4c9d010e5fcf97082d62cb54415bf6cf0604c" gracePeriod=45
Sep 19 11:01:22.490262 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.490236    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:01:22.586350 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.586282    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:01:22.651550 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.651511    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-home\") pod \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") "
Sep 19 11:01:22.651665 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.651563    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-dshm\") pod \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") "
Sep 19 11:01:22.651665 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.651589    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4qdk\" (UniqueName: \"kubernetes.io/projected/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-kube-api-access-f4qdk\") pod \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") "
Sep 19 11:01:22.651665 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.651611    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-model-cache\") pod \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") "
Sep 19 11:01:22.651665 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.651640    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-tmp-dir\") pod \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") "
Sep 19 11:01:22.651889 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.651678    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-tls-certs\") pod \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\" (UID: \"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88\") "
Sep 19 11:01:22.651889 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.651780    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-home" (OuterVolumeSpecName: "home") pod "36fbe2ce-68cc-45cf-9cf0-94f272ae1f88" (UID: "36fbe2ce-68cc-45cf-9cf0-94f272ae1f88"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:01:22.651991 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.651907    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "36fbe2ce-68cc-45cf-9cf0-94f272ae1f88" (UID: "36fbe2ce-68cc-45cf-9cf0-94f272ae1f88"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:01:22.651991 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.651928    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-model-cache" (OuterVolumeSpecName: "model-cache") pod "36fbe2ce-68cc-45cf-9cf0-94f272ae1f88" (UID: "36fbe2ce-68cc-45cf-9cf0-94f272ae1f88"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:01:22.651991 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.651945    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:01:22.653789 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.653767    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-dshm" (OuterVolumeSpecName: "dshm") pod "36fbe2ce-68cc-45cf-9cf0-94f272ae1f88" (UID: "36fbe2ce-68cc-45cf-9cf0-94f272ae1f88"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:01:22.653911 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.653798    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "36fbe2ce-68cc-45cf-9cf0-94f272ae1f88" (UID: "36fbe2ce-68cc-45cf-9cf0-94f272ae1f88"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:01:22.653911 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.653830    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-kube-api-access-f4qdk" (OuterVolumeSpecName: "kube-api-access-f4qdk") pod "36fbe2ce-68cc-45cf-9cf0-94f272ae1f88" (UID: "36fbe2ce-68cc-45cf-9cf0-94f272ae1f88"). InnerVolumeSpecName "kube-api-access-f4qdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:01:22.752477 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.752451    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:01:22.752477 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.752472    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:01:22.752477 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.752480    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4qdk\" (UniqueName: \"kubernetes.io/projected/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-kube-api-access-f4qdk\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:01:22.752743 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.752490    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:01:22.752743 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:22.752498    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:01:23.265516 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.265469    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m" podUID="2b3e5364-f781-4a27-aee9-b69fcefc556f" containerName="main" containerID="cri-o://389ae259e797b5704a93fc3c096d4b04a607edb9756d8528a9239bb2d16ae2ad" gracePeriod=45
Sep 19 11:01:23.315922 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.315896    2579 generic.go:356] "Generic (PLEG): container finished" podID="36fbe2ce-68cc-45cf-9cf0-94f272ae1f88" containerID="3eeea7dcd091f771d812d02c74d4c9d010e5fcf97082d62cb54415bf6cf0604c" exitCode=0
Sep 19 11:01:23.316032 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.315932    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"
Sep 19 11:01:23.316032 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.315942    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld" event={"ID":"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88","Type":"ContainerDied","Data":"3eeea7dcd091f771d812d02c74d4c9d010e5fcf97082d62cb54415bf6cf0604c"}
Sep 19 11:01:23.316032 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.315973    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld" event={"ID":"36fbe2ce-68cc-45cf-9cf0-94f272ae1f88","Type":"ContainerDied","Data":"3fd5857de2140575747d4e3556701b5ea9209d19bf3873d9dc3c329df6b98d6d"}
Sep 19 11:01:23.316032 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.315990    2579 scope.go:117] "RemoveContainer" containerID="3eeea7dcd091f771d812d02c74d4c9d010e5fcf97082d62cb54415bf6cf0604c"
Sep 19 11:01:23.340148 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.340121    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"]
Sep 19 11:01:23.344054 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.344034    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-72e262/rate-test-model-72e262-kserve-5f9dd88bdc-zqwld"]
Sep 19 11:01:23.375952 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.375936    2579 scope.go:117] "RemoveContainer" containerID="3eeea7dcd091f771d812d02c74d4c9d010e5fcf97082d62cb54415bf6cf0604c"
Sep 19 11:01:23.376293 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:01:23.376273    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3eeea7dcd091f771d812d02c74d4c9d010e5fcf97082d62cb54415bf6cf0604c\": container with ID starting with 3eeea7dcd091f771d812d02c74d4c9d010e5fcf97082d62cb54415bf6cf0604c not found: ID does not exist" containerID="3eeea7dcd091f771d812d02c74d4c9d010e5fcf97082d62cb54415bf6cf0604c"
Sep 19 11:01:23.376400 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.376299    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3eeea7dcd091f771d812d02c74d4c9d010e5fcf97082d62cb54415bf6cf0604c"} err="failed to get container status \"3eeea7dcd091f771d812d02c74d4c9d010e5fcf97082d62cb54415bf6cf0604c\": rpc error: code = NotFound desc = could not find container \"3eeea7dcd091f771d812d02c74d4c9d010e5fcf97082d62cb54415bf6cf0604c\": container with ID starting with 3eeea7dcd091f771d812d02c74d4c9d010e5fcf97082d62cb54415bf6cf0604c not found: ID does not exist"
Sep 19 11:01:23.505557 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.505534    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:01:23.659813 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.659742    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-tmp-dir\") pod \"2b3e5364-f781-4a27-aee9-b69fcefc556f\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") "
Sep 19 11:01:23.659813 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.659796    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-dshm\") pod \"2b3e5364-f781-4a27-aee9-b69fcefc556f\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") "
Sep 19 11:01:23.660027 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.659825    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b3e5364-f781-4a27-aee9-b69fcefc556f-tls-certs\") pod \"2b3e5364-f781-4a27-aee9-b69fcefc556f\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") "
Sep 19 11:01:23.660027 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.659874    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-home\") pod \"2b3e5364-f781-4a27-aee9-b69fcefc556f\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") "
Sep 19 11:01:23.660027 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.659897    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvlmw\" (UniqueName: \"kubernetes.io/projected/2b3e5364-f781-4a27-aee9-b69fcefc556f-kube-api-access-fvlmw\") pod \"2b3e5364-f781-4a27-aee9-b69fcefc556f\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") "
Sep 19 11:01:23.660027 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.659964    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-model-cache\") pod \"2b3e5364-f781-4a27-aee9-b69fcefc556f\" (UID: \"2b3e5364-f781-4a27-aee9-b69fcefc556f\") "
Sep 19 11:01:23.660246 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.660028    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2b3e5364-f781-4a27-aee9-b69fcefc556f" (UID: "2b3e5364-f781-4a27-aee9-b69fcefc556f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:01:23.660246 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.660119    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-home" (OuterVolumeSpecName: "home") pod "2b3e5364-f781-4a27-aee9-b69fcefc556f" (UID: "2b3e5364-f781-4a27-aee9-b69fcefc556f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:01:23.660246 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.660224    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-model-cache" (OuterVolumeSpecName: "model-cache") pod "2b3e5364-f781-4a27-aee9-b69fcefc556f" (UID: "2b3e5364-f781-4a27-aee9-b69fcefc556f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:01:23.660350 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.660287    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:01:23.660350 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.660302    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:01:23.660350 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.660320    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:01:23.662030 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.661997    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b3e5364-f781-4a27-aee9-b69fcefc556f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2b3e5364-f781-4a27-aee9-b69fcefc556f" (UID: "2b3e5364-f781-4a27-aee9-b69fcefc556f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:01:23.662030 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.662018    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-dshm" (OuterVolumeSpecName: "dshm") pod "2b3e5364-f781-4a27-aee9-b69fcefc556f" (UID: "2b3e5364-f781-4a27-aee9-b69fcefc556f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:01:23.662180 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.662032    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b3e5364-f781-4a27-aee9-b69fcefc556f-kube-api-access-fvlmw" (OuterVolumeSpecName: "kube-api-access-fvlmw") pod "2b3e5364-f781-4a27-aee9-b69fcefc556f" (UID: "2b3e5364-f781-4a27-aee9-b69fcefc556f"). InnerVolumeSpecName "kube-api-access-fvlmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:01:23.761559 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.761518    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvlmw\" (UniqueName: \"kubernetes.io/projected/2b3e5364-f781-4a27-aee9-b69fcefc556f-kube-api-access-fvlmw\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:01:23.761559 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.761555    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b3e5364-f781-4a27-aee9-b69fcefc556f-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:01:23.761695 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:23.761564    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b3e5364-f781-4a27-aee9-b69fcefc556f-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:01:24.322719 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:24.322685    2579 generic.go:356] "Generic (PLEG): container finished" podID="2b3e5364-f781-4a27-aee9-b69fcefc556f" containerID="389ae259e797b5704a93fc3c096d4b04a607edb9756d8528a9239bb2d16ae2ad" exitCode=0
Sep 19 11:01:24.323142 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:24.322726    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"
Sep 19 11:01:24.323142 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:24.322767    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m" event={"ID":"2b3e5364-f781-4a27-aee9-b69fcefc556f","Type":"ContainerDied","Data":"389ae259e797b5704a93fc3c096d4b04a607edb9756d8528a9239bb2d16ae2ad"}
Sep 19 11:01:24.323142 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:24.322809    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m" event={"ID":"2b3e5364-f781-4a27-aee9-b69fcefc556f","Type":"ContainerDied","Data":"41928698dd2bacb901a619be5b119d3428b6f806a20623d4f293627f91b180d4"}
Sep 19 11:01:24.323142 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:24.322826    2579 scope.go:117] "RemoveContainer" containerID="389ae259e797b5704a93fc3c096d4b04a607edb9756d8528a9239bb2d16ae2ad"
Sep 19 11:01:24.332276 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:24.332246    2579 scope.go:117] "RemoveContainer" containerID="389ae259e797b5704a93fc3c096d4b04a607edb9756d8528a9239bb2d16ae2ad"
Sep 19 11:01:24.332535 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:01:24.332504    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"389ae259e797b5704a93fc3c096d4b04a607edb9756d8528a9239bb2d16ae2ad\": container with ID starting with 389ae259e797b5704a93fc3c096d4b04a607edb9756d8528a9239bb2d16ae2ad not found: ID does not exist" containerID="389ae259e797b5704a93fc3c096d4b04a607edb9756d8528a9239bb2d16ae2ad"
Sep 19 11:01:24.332671 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:24.332646    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"389ae259e797b5704a93fc3c096d4b04a607edb9756d8528a9239bb2d16ae2ad"} err="failed to get container status \"389ae259e797b5704a93fc3c096d4b04a607edb9756d8528a9239bb2d16ae2ad\": rpc error: code = NotFound desc = could not find container \"389ae259e797b5704a93fc3c096d4b04a607edb9756d8528a9239bb2d16ae2ad\": container with ID starting with 389ae259e797b5704a93fc3c096d4b04a607edb9756d8528a9239bb2d16ae2ad not found: ID does not exist"
Sep 19 11:01:24.342830 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:24.342810    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"]
Sep 19 11:01:24.346604 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:24.346579    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99b43e/rate-test-model-99b43e-kserve-b7f7dcb5f-ksr6m"]
Sep 19 11:01:25.115240 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:25.115204    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b3e5364-f781-4a27-aee9-b69fcefc556f" path="/var/lib/kubelet/pods/2b3e5364-f781-4a27-aee9-b69fcefc556f/volumes"
Sep 19 11:01:25.115618 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:25.115603    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36fbe2ce-68cc-45cf-9cf0-94f272ae1f88" path="/var/lib/kubelet/pods/36fbe2ce-68cc-45cf-9cf0-94f272ae1f88/volumes"
Sep 19 11:01:35.251820 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:35.251784    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:35.755667 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:35.755636    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:36.340598 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:36.340571    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:36.624963 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:36.624894    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:37.070543 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:37.070500    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:37.281683 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:37.281656    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:37.602907 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:37.602873    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:37.745946 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:37.745917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:40.185586 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:40.185557    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:40.784992 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:40.784958    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:40.888033 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:40.887998    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:40.990607 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:40.990580    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:41.570493 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:41.570460    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:42.265584 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.265543    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:42.757541 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.757502    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"]
Sep 19 11:01:42.757897 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.757867    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b3e5364-f781-4a27-aee9-b69fcefc556f" containerName="main"
Sep 19 11:01:42.757897 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.757883    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b3e5364-f781-4a27-aee9-b69fcefc556f" containerName="main"
Sep 19 11:01:42.757897 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.757897    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36fbe2ce-68cc-45cf-9cf0-94f272ae1f88" containerName="main"
Sep 19 11:01:42.758008 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.757903    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="36fbe2ce-68cc-45cf-9cf0-94f272ae1f88" containerName="main"
Sep 19 11:01:42.758008 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.757972    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="36fbe2ce-68cc-45cf-9cf0-94f272ae1f88" containerName="main"
Sep 19 11:01:42.758008 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.757983    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b3e5364-f781-4a27-aee9-b69fcefc556f" containerName="main"
Sep 19 11:01:42.762125 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.762105    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:42.764944 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.764922    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-f210d9-serving-cert\""
Sep 19 11:01:42.771985 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.771963    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"]
Sep 19 11:01:42.796067 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.796044    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:42.814299 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.814275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fc8282b-5810-45a0-885d-47a0915b3a68-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:42.814392 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.814308    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm9ww\" (UniqueName: \"kubernetes.io/projected/8fc8282b-5810-45a0-885d-47a0915b3a68-kube-api-access-xm9ww\") pod \"maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:42.814392 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.814329    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-metrics-certs\") pod \"maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:42.814470 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.814422    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:42.893472 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.893451    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:42.915589 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.915564    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:42.915709 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.915629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fc8282b-5810-45a0-885d-47a0915b3a68-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:42.915709 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.915669    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm9ww\" (UniqueName: \"kubernetes.io/projected/8fc8282b-5810-45a0-885d-47a0915b3a68-kube-api-access-xm9ww\") pod \"maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:42.915709 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:01:42.915693    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-serving-cert: secret "maas-api-e2e-ipp-b-f210d9-serving-cert" not found
Sep 19 11:01:42.915709 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.915696    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-metrics-certs\") pod \"maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:42.915935 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:01:42.915752    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-maas-api-tls podName:8fc8282b-5810-45a0-885d-47a0915b3a68 nodeName:}" failed. No retries permitted until 2026-09-19 11:01:43.415737147 +0000 UTC m=+1624.327430649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-maas-api-tls") pod "maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn" (UID: "8fc8282b-5810-45a0-885d-47a0915b3a68") : secret "maas-api-e2e-ipp-b-f210d9-serving-cert" not found
Sep 19 11:01:42.916278 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.916260    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fc8282b-5810-45a0-885d-47a0915b3a68-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:42.918096 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.918076    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-metrics-certs\") pod \"maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:42.924663 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:42.924639    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm9ww\" (UniqueName: \"kubernetes.io/projected/8fc8282b-5810-45a0-885d-47a0915b3a68-kube-api-access-xm9ww\") pod \"maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:43.026575 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:43.026485    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:43.229417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:43.229383    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:43.323184 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:43.323104    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:01:43.420709 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:43.420671    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:43.423032 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:43.423014    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:43.675669 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:43.675638    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:43.947162 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:43.947135    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"]
Sep 19 11:01:43.948361 ip-10-0-140-105 kubenswrapper[2579]: W0919 11:01:43.948335    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fc8282b_5810_45a0_885d_47a0915b3a68.slice/crio-b48cf7cdfbb8ffc1c063871c9f066e31508c27aade74950fe7691d8d5993b3f9 WatchSource:0}: Error finding container b48cf7cdfbb8ffc1c063871c9f066e31508c27aade74950fe7691d8d5993b3f9: Status 404 returned error can't find the container with id b48cf7cdfbb8ffc1c063871c9f066e31508c27aade74950fe7691d8d5993b3f9
Sep 19 11:01:44.424866 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:44.424841    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn" event={"ID":"8fc8282b-5810-45a0-885d-47a0915b3a68","Type":"ContainerStarted","Data":"e24c769224da6cdbcba4d42f35916b00706b67db7a75739ddf21debf0ac7b8a4"}
Sep 19 11:01:44.424987 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:44.424886    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn" event={"ID":"8fc8282b-5810-45a0-885d-47a0915b3a68","Type":"ContainerStarted","Data":"b48cf7cdfbb8ffc1c063871c9f066e31508c27aade74950fe7691d8d5993b3f9"}
Sep 19 11:01:44.424987 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:44.424927    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:44.442604 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:44.442561    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn" podStartSLOduration=2.062731844 podStartE2EDuration="2.442548629s" totalImagesPullingTime="379.816785ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:01:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:01:44.440155273 +0000 UTC m=+1625.351848792" watchObservedRunningTime="2026-09-19 11:01:44.442548629 +0000 UTC m=+1625.354242139"
Sep 19 11:01:48.000982 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:48.000943    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:01:50.436633 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:50.436604    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:01:58.268903 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:01:58.268869    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:04.064953 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:04.064924    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:05.478016 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:05.477979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:05.817704 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:05.817605    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:06.181576 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:06.181547    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:06.843765 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:06.843723    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:07.359980 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:07.359942    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:07.955602 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:07.955566    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:13.841588 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:13.841553    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"]
Sep 19 11:02:13.845192 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:13.845175    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:13.847853 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:13.847832    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-bd3aad\"/\"openshift-service-ca.crt\""
Sep 19 11:02:13.849141 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:13.849119    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-bd3aad\"/\"default-dockercfg-8hg5n\""
Sep 19 11:02:13.849251 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:13.849142    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-bd3aad\"/\"kube-root-ca.crt\""
Sep 19 11:02:13.849251 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:13.849142    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-bd3aad\"/\"ipp-route-bd3aad-kserve-self-signed-certs\""
Sep 19 11:02:13.854951 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:13.854931    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"]
Sep 19 11:02:13.870832 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:13.870812    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:13.963684 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:13.963655    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce267735-7ee2-460e-8d42-90dfc136118a-tls-certs\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:13.963684 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:13.963684    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-home\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:13.963882 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:13.963719    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-tmp-dir\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:13.963882 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:13.963833    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7669x\" (UniqueName: \"kubernetes.io/projected/ce267735-7ee2-460e-8d42-90dfc136118a-kube-api-access-7669x\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:13.963997 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:13.963920    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-dshm\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:13.963997 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:13.963955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-model-cache\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:14.064867 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.064833    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce267735-7ee2-460e-8d42-90dfc136118a-tls-certs\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:14.065014 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.064870    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-home\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:14.065014 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.064898    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-tmp-dir\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:14.065014 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.064933    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7669x\" (UniqueName: \"kubernetes.io/projected/ce267735-7ee2-460e-8d42-90dfc136118a-kube-api-access-7669x\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:14.065014 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.064973    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-dshm\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:14.065014 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.064995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-model-cache\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:14.065360 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.065338    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-home\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:14.065425 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.065382    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-model-cache\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:14.065425 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.065414    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-tmp-dir\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:14.067076 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.067054    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-dshm\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:14.067323 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.067309    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce267735-7ee2-460e-8d42-90dfc136118a-tls-certs\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:14.072353 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.072337    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7669x\" (UniqueName: \"kubernetes.io/projected/ce267735-7ee2-460e-8d42-90dfc136118a-kube-api-access-7669x\") pod \"ipp-route-bd3aad-kserve-686d8bf955-cn864\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") " pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:14.156068 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.155997    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:14.424376 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.424346    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"]
Sep 19 11:02:14.425698 ip-10-0-140-105 kubenswrapper[2579]: W0919 11:02:14.425670    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce267735_7ee2_460e_8d42_90dfc136118a.slice/crio-d83ed9d194c7e5d4e350e186fb4cf86b27b5927d844e59e98e24e3dc97b620e7 WatchSource:0}: Error finding container d83ed9d194c7e5d4e350e186fb4cf86b27b5927d844e59e98e24e3dc97b620e7: Status 404 returned error can't find the container with id d83ed9d194c7e5d4e350e186fb4cf86b27b5927d844e59e98e24e3dc97b620e7
Sep 19 11:02:14.584400 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.584365    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864" event={"ID":"ce267735-7ee2-460e-8d42-90dfc136118a","Type":"ContainerStarted","Data":"98f6ba46b3ec9207d1699c482f86619a1e209abe2399cc1d1956226bc38582b2"}
Sep 19 11:02:14.584594 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.584410    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864" event={"ID":"ce267735-7ee2-460e-8d42-90dfc136118a","Type":"ContainerStarted","Data":"d83ed9d194c7e5d4e350e186fb4cf86b27b5927d844e59e98e24e3dc97b620e7"}
Sep 19 11:02:14.601098 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:14.601055    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864" podStartSLOduration=1.601041197 podStartE2EDuration="1.601041197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:02:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:02:14.600105882 +0000 UTC m=+1655.511799403" watchObservedRunningTime="2026-09-19 11:02:14.601041197 +0000 UTC m=+1655.512734716"
Sep 19 11:02:17.989897 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:17.989858    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:02:24.157154 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:24.157116    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:24.157637 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:24.157172    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:24.169615 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:24.169588    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:24.645594 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:24.645568    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:02:28.721516 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:28.721481    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:28.821644 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:28.821604    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:32.989178 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:32.989143    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:02:38.142562 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:38.142512    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:39.471729 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:39.471685    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:39.775145 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:39.775063    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:39.995398 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:39.995365    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:42.473258 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:42.473212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:43.136401 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:43.136371    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:43.236766 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:43.236737    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:43.337770 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:43.337741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:44.067427 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:44.067395    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:44.669412 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:44.669383    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:45.453198 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:45.453163    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:45.953726 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:45.953693    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"]
Sep 19 11:02:45.957566 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:45.957546    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:45.959993 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:45.959974    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-953b43-serving-cert\""
Sep 19 11:02:45.969710 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:45.969683    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"]
Sep 19 11:02:45.981792 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:45.981766    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:46.036108 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.036084    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:46.043304 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.043286    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:46.043381 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.043315    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:46.043381 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.043337    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ftg4\" (UniqueName: \"kubernetes.io/projected/b80f0a37-607a-45f4-98ff-f5442875988e-kube-api-access-7ftg4\") pod \"maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:46.043490 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.043470    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b80f0a37-607a-45f4-98ff-f5442875988e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:46.144748 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.144723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:46.144854 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.144753    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:46.144854 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.144770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ftg4\" (UniqueName: \"kubernetes.io/projected/b80f0a37-607a-45f4-98ff-f5442875988e-kube-api-access-7ftg4\") pod \"maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:46.144854 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.144849    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b80f0a37-607a-45f4-98ff-f5442875988e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:46.145008 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:02:46.144900    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-serving-cert: secret "maas-api-e2e-ipp-cleanup-953b43-serving-cert" not found
Sep 19 11:02:46.145008 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:02:46.144969    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-maas-api-tls podName:b80f0a37-607a-45f4-98ff-f5442875988e nodeName:}" failed. No retries permitted until 2026-09-19 11:02:46.644951144 +0000 UTC m=+1687.556644641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th" (UID: "b80f0a37-607a-45f4-98ff-f5442875988e") : secret "maas-api-e2e-ipp-cleanup-953b43-serving-cert" not found
Sep 19 11:02:46.145546 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.145514    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b80f0a37-607a-45f4-98ff-f5442875988e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:46.147158 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.147141    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:46.153093 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.153072    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ftg4\" (UniqueName: \"kubernetes.io/projected/b80f0a37-607a-45f4-98ff-f5442875988e-kube-api-access-7ftg4\") pod \"maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:46.279568 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.279457    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:46.414573 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.414519    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:46.511079 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.511045    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:02:46.650605 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.650505    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:46.652895 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.652877    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:46.869715 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:46.869684    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:47.142843 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:47.142814    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"]
Sep 19 11:02:47.145058 ip-10-0-140-105 kubenswrapper[2579]: W0919 11:02:47.145033    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb80f0a37_607a_45f4_98ff_f5442875988e.slice/crio-1de957d893de0885996dbe481ed83c64177c8c5466672da4943feb07bd64a5f5 WatchSource:0}: Error finding container 1de957d893de0885996dbe481ed83c64177c8c5466672da4943feb07bd64a5f5: Status 404 returned error can't find the container with id 1de957d893de0885996dbe481ed83c64177c8c5466672da4943feb07bd64a5f5
Sep 19 11:02:47.757780 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:47.757692    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th" event={"ID":"b80f0a37-607a-45f4-98ff-f5442875988e","Type":"ContainerStarted","Data":"ebbd9e323e69a1ea2c7a904347573f86d82d7e773699d0e5cac34a8d504dd9f4"}
Sep 19 11:02:47.757780 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:47.757741    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th" event={"ID":"b80f0a37-607a-45f4-98ff-f5442875988e","Type":"ContainerStarted","Data":"1de957d893de0885996dbe481ed83c64177c8c5466672da4943feb07bd64a5f5"}
Sep 19 11:02:47.758181 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:47.757794    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:02:47.775604 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:47.775558    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th" podStartSLOduration=2.431679386 podStartE2EDuration="2.775545728s" totalImagesPullingTime="343.866342ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:02:47.772860504 +0000 UTC m=+1688.684554026" watchObservedRunningTime="2026-09-19 11:02:47.775545728 +0000 UTC m=+1688.687239247"
Sep 19 11:02:52.895494 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:52.895454    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:02:53.769339 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:02:53.769269    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:03:08.150220 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:08.150186    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:08.305910 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:08.305882    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:09.243577 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:09.243543    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:09.967226 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:09.967199    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:10.652486 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:10.652451    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:11.236007 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:11.235973    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:11.770274 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:11.770230    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:12.356205 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:12.356178    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:13.066963 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:13.066928    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:14.468789 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:14.468754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:14.962594 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:14.962561    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"]
Sep 19 11:03:14.962838 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:14.962809    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th" podUID="b80f0a37-607a-45f4-98ff-f5442875988e" containerName="maas-api" containerID="cri-o://ebbd9e323e69a1ea2c7a904347573f86d82d7e773699d0e5cac34a8d504dd9f4" gracePeriod=30
Sep 19 11:03:14.991112 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:14.991091    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:15.092483 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.092453    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:15.153577 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.153549    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:15.205084 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.205063    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:03:15.213574 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.213506    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:15.301991 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.301952    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b80f0a37-607a-45f4-98ff-f5442875988e-openshift-service-ca\") pod \"b80f0a37-607a-45f4-98ff-f5442875988e\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") "
Sep 19 11:03:15.302139 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.302012    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-maas-api-tls\") pod \"b80f0a37-607a-45f4-98ff-f5442875988e\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") "
Sep 19 11:03:15.302139 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.302072    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-metrics-certs\") pod \"b80f0a37-607a-45f4-98ff-f5442875988e\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") "
Sep 19 11:03:15.302139 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.302108    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ftg4\" (UniqueName: \"kubernetes.io/projected/b80f0a37-607a-45f4-98ff-f5442875988e-kube-api-access-7ftg4\") pod \"b80f0a37-607a-45f4-98ff-f5442875988e\" (UID: \"b80f0a37-607a-45f4-98ff-f5442875988e\") "
Sep 19 11:03:15.302308 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.302291    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b80f0a37-607a-45f4-98ff-f5442875988e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b80f0a37-607a-45f4-98ff-f5442875988e" (UID: "b80f0a37-607a-45f4-98ff-f5442875988e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:03:15.304144 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.304114    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b80f0a37-607a-45f4-98ff-f5442875988e" (UID: "b80f0a37-607a-45f4-98ff-f5442875988e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:03:15.304242 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.304204    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b80f0a37-607a-45f4-98ff-f5442875988e-kube-api-access-7ftg4" (OuterVolumeSpecName: "kube-api-access-7ftg4") pod "b80f0a37-607a-45f4-98ff-f5442875988e" (UID: "b80f0a37-607a-45f4-98ff-f5442875988e"). InnerVolumeSpecName "kube-api-access-7ftg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:03:15.304308 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.304294    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b80f0a37-607a-45f4-98ff-f5442875988e" (UID: "b80f0a37-607a-45f4-98ff-f5442875988e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:03:15.403444 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.403404    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b80f0a37-607a-45f4-98ff-f5442875988e-openshift-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:03:15.403444 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.403441    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-maas-api-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:03:15.403444 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.403452    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b80f0a37-607a-45f4-98ff-f5442875988e-metrics-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:03:15.403750 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.403460    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ftg4\" (UniqueName: \"kubernetes.io/projected/b80f0a37-607a-45f4-98ff-f5442875988e-kube-api-access-7ftg4\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:03:15.528584 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.528487    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:15.626852 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.626824    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:15.910027 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.909918    2579 generic.go:356] "Generic (PLEG): container finished" podID="b80f0a37-607a-45f4-98ff-f5442875988e" containerID="ebbd9e323e69a1ea2c7a904347573f86d82d7e773699d0e5cac34a8d504dd9f4" exitCode=0
Sep 19 11:03:15.910027 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.909971    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"
Sep 19 11:03:15.910027 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.910003    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th" event={"ID":"b80f0a37-607a-45f4-98ff-f5442875988e","Type":"ContainerDied","Data":"ebbd9e323e69a1ea2c7a904347573f86d82d7e773699d0e5cac34a8d504dd9f4"}
Sep 19 11:03:15.910425 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.910089    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th" event={"ID":"b80f0a37-607a-45f4-98ff-f5442875988e","Type":"ContainerDied","Data":"1de957d893de0885996dbe481ed83c64177c8c5466672da4943feb07bd64a5f5"}
Sep 19 11:03:15.910425 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.910120    2579 scope.go:117] "RemoveContainer" containerID="ebbd9e323e69a1ea2c7a904347573f86d82d7e773699d0e5cac34a8d504dd9f4"
Sep 19 11:03:15.920453 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.920438    2579 scope.go:117] "RemoveContainer" containerID="ebbd9e323e69a1ea2c7a904347573f86d82d7e773699d0e5cac34a8d504dd9f4"
Sep 19 11:03:15.920772 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:03:15.920752    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebbd9e323e69a1ea2c7a904347573f86d82d7e773699d0e5cac34a8d504dd9f4\": container with ID starting with ebbd9e323e69a1ea2c7a904347573f86d82d7e773699d0e5cac34a8d504dd9f4 not found: ID does not exist" containerID="ebbd9e323e69a1ea2c7a904347573f86d82d7e773699d0e5cac34a8d504dd9f4"
Sep 19 11:03:15.920833 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.920781    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebbd9e323e69a1ea2c7a904347573f86d82d7e773699d0e5cac34a8d504dd9f4"} err="failed to get container status \"ebbd9e323e69a1ea2c7a904347573f86d82d7e773699d0e5cac34a8d504dd9f4\": rpc error: code = NotFound desc = could not find container \"ebbd9e323e69a1ea2c7a904347573f86d82d7e773699d0e5cac34a8d504dd9f4\": container with ID starting with ebbd9e323e69a1ea2c7a904347573f86d82d7e773699d0e5cac34a8d504dd9f4 not found: ID does not exist"
Sep 19 11:03:15.931079 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.931059    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"]
Sep 19 11:03:15.934649 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:15.934629    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-953b43-66c756996-hc7th"]
Sep 19 11:03:16.255210 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:16.255178    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:16.875337 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:16.875304    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:17.115494 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:17.115465    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b80f0a37-607a-45f4-98ff-f5442875988e" path="/var/lib/kubelet/pods/b80f0a37-607a-45f4-98ff-f5442875988e/volumes"
Sep 19 11:03:17.467347 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:17.467292    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:17.723776 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:17.723698    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:17.823967 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:17.823937    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:18.065189 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:18.065108    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:19.387999 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:19.387968    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:03:21.246550 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:21.246501    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:21.343991 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:21.343955    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:21.491671 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:21.491640    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:25.135053 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:25.135021    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:25.788087 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:25.788057    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:26.567078 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:26.567044    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:27.169197 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:27.169165    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:27.756100 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:27.756068    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:28.371311 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:28.371277    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:38.193373 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:38.193337    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:03:40.377921 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:40.377891    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:40.440422 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:40.440387    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:40.500848 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:40.500818    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:40.560449 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:40.560423    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:40.912920 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:40.912889    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:41.013479 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:41.013451    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:41.877671 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:41.877639    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:42.479455 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:42.479422    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:43.004215 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:43.004180    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:45.893895 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:45.893860    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"]
Sep 19 11:03:45.991175 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:45.991144    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:03:50.386822 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:03:50.386783    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:04:00.914711 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:00.914671    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864" podUID="ce267735-7ee2-460e-8d42-90dfc136118a" containerName="main" containerID="cri-o://98f6ba46b3ec9207d1699c482f86619a1e209abe2399cc1d1956226bc38582b2" gracePeriod=45
Sep 19 11:04:01.147236 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.147209    2579 generic.go:356] "Generic (PLEG): container finished" podID="ce267735-7ee2-460e-8d42-90dfc136118a" containerID="98f6ba46b3ec9207d1699c482f86619a1e209abe2399cc1d1956226bc38582b2" exitCode=0
Sep 19 11:04:01.147357 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.147278    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864" event={"ID":"ce267735-7ee2-460e-8d42-90dfc136118a","Type":"ContainerDied","Data":"98f6ba46b3ec9207d1699c482f86619a1e209abe2399cc1d1956226bc38582b2"}
Sep 19 11:04:01.165834 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.165787    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:04:01.284116 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.284083    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-home\") pod \"ce267735-7ee2-460e-8d42-90dfc136118a\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") "
Sep 19 11:04:01.284116 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.284119    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce267735-7ee2-460e-8d42-90dfc136118a-tls-certs\") pod \"ce267735-7ee2-460e-8d42-90dfc136118a\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") "
Sep 19 11:04:01.284354 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.284141    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-tmp-dir\") pod \"ce267735-7ee2-460e-8d42-90dfc136118a\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") "
Sep 19 11:04:01.284354 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.284162    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-dshm\") pod \"ce267735-7ee2-460e-8d42-90dfc136118a\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") "
Sep 19 11:04:01.284354 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.284204    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-model-cache\") pod \"ce267735-7ee2-460e-8d42-90dfc136118a\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") "
Sep 19 11:04:01.284354 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.284245    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7669x\" (UniqueName: \"kubernetes.io/projected/ce267735-7ee2-460e-8d42-90dfc136118a-kube-api-access-7669x\") pod \"ce267735-7ee2-460e-8d42-90dfc136118a\" (UID: \"ce267735-7ee2-460e-8d42-90dfc136118a\") "
Sep 19 11:04:01.284579 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.284442    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ce267735-7ee2-460e-8d42-90dfc136118a" (UID: "ce267735-7ee2-460e-8d42-90dfc136118a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:04:01.284579 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.284460    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-home" (OuterVolumeSpecName: "home") pod "ce267735-7ee2-460e-8d42-90dfc136118a" (UID: "ce267735-7ee2-460e-8d42-90dfc136118a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:04:01.284579 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.284502    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-model-cache" (OuterVolumeSpecName: "model-cache") pod "ce267735-7ee2-460e-8d42-90dfc136118a" (UID: "ce267735-7ee2-460e-8d42-90dfc136118a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:04:01.286368 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.286345    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-dshm" (OuterVolumeSpecName: "dshm") pod "ce267735-7ee2-460e-8d42-90dfc136118a" (UID: "ce267735-7ee2-460e-8d42-90dfc136118a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:04:01.286485 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.286361    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce267735-7ee2-460e-8d42-90dfc136118a-kube-api-access-7669x" (OuterVolumeSpecName: "kube-api-access-7669x") pod "ce267735-7ee2-460e-8d42-90dfc136118a" (UID: "ce267735-7ee2-460e-8d42-90dfc136118a"). InnerVolumeSpecName "kube-api-access-7669x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:04:01.286485 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.286409    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce267735-7ee2-460e-8d42-90dfc136118a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ce267735-7ee2-460e-8d42-90dfc136118a" (UID: "ce267735-7ee2-460e-8d42-90dfc136118a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:04:01.385774 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.385741    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:04:01.385774 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.385766    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:04:01.385774 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.385775    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:04:01.385774 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.385784    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7669x\" (UniqueName: \"kubernetes.io/projected/ce267735-7ee2-460e-8d42-90dfc136118a-kube-api-access-7669x\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:04:01.386036 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.385792    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce267735-7ee2-460e-8d42-90dfc136118a-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:04:01.386036 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:01.385799    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce267735-7ee2-460e-8d42-90dfc136118a-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:04:02.153634 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:02.153605    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"
Sep 19 11:04:02.154104 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:02.153644    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864" event={"ID":"ce267735-7ee2-460e-8d42-90dfc136118a","Type":"ContainerDied","Data":"d83ed9d194c7e5d4e350e186fb4cf86b27b5927d844e59e98e24e3dc97b620e7"}
Sep 19 11:04:02.154104 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:02.153702    2579 scope.go:117] "RemoveContainer" containerID="98f6ba46b3ec9207d1699c482f86619a1e209abe2399cc1d1956226bc38582b2"
Sep 19 11:04:02.174847 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:02.174820    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"]
Sep 19 11:04:02.178495 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:02.178470    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-bd3aad/ipp-route-bd3aad-kserve-686d8bf955-cn864"]
Sep 19 11:04:03.115619 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:03.115588    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce267735-7ee2-460e-8d42-90dfc136118a" path="/var/lib/kubelet/pods/ce267735-7ee2-460e-8d42-90dfc136118a/volumes"
Sep 19 11:04:06.505163 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:06.505131    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:22.378206 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:22.378177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:23.229570 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:23.229542    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:28.805153 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:28.805120    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:28.868875 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:28.868846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:29.068771 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:29.068686    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:29.766477 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:29.766443    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:30.469410 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:30.469373    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:30.969076 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:30.969044    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:31.577999 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:31.577966    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:32.374328 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:32.374293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:39.873910 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:39.873879    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"]
Sep 19 11:04:39.874360 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:39.874125    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn" podUID="8fc8282b-5810-45a0-885d-47a0915b3a68" containerName="maas-api" containerID="cri-o://e24c769224da6cdbcba4d42f35916b00706b67db7a75739ddf21debf0ac7b8a4" gracePeriod=30
Sep 19 11:04:39.904708 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:39.904681    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:39.964690 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:39.964659    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:40.070403 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.070372    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:40.114414 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.114393    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:04:40.135051 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.134986    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:40.199650 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.199627    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-metrics-certs\") pod \"8fc8282b-5810-45a0-885d-47a0915b3a68\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") "
Sep 19 11:04:40.199792 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.199680    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fc8282b-5810-45a0-885d-47a0915b3a68-openshift-service-ca\") pod \"8fc8282b-5810-45a0-885d-47a0915b3a68\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") "
Sep 19 11:04:40.199792 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.199737    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm9ww\" (UniqueName: \"kubernetes.io/projected/8fc8282b-5810-45a0-885d-47a0915b3a68-kube-api-access-xm9ww\") pod \"8fc8282b-5810-45a0-885d-47a0915b3a68\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") "
Sep 19 11:04:40.199792 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.199754    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-maas-api-tls\") pod \"8fc8282b-5810-45a0-885d-47a0915b3a68\" (UID: \"8fc8282b-5810-45a0-885d-47a0915b3a68\") "
Sep 19 11:04:40.200087 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.200060    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fc8282b-5810-45a0-885d-47a0915b3a68-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8fc8282b-5810-45a0-885d-47a0915b3a68" (UID: "8fc8282b-5810-45a0-885d-47a0915b3a68"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:04:40.201813 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.201793    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8fc8282b-5810-45a0-885d-47a0915b3a68" (UID: "8fc8282b-5810-45a0-885d-47a0915b3a68"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:04:40.202248 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.202224    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8fc8282b-5810-45a0-885d-47a0915b3a68" (UID: "8fc8282b-5810-45a0-885d-47a0915b3a68"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:04:40.202337 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.202255    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fc8282b-5810-45a0-885d-47a0915b3a68-kube-api-access-xm9ww" (OuterVolumeSpecName: "kube-api-access-xm9ww") pod "8fc8282b-5810-45a0-885d-47a0915b3a68" (UID: "8fc8282b-5810-45a0-885d-47a0915b3a68"). InnerVolumeSpecName "kube-api-access-xm9ww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:04:40.300878 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.300851    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-metrics-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:04:40.300878 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.300873    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fc8282b-5810-45a0-885d-47a0915b3a68-openshift-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:04:40.300878 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.300882    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm9ww\" (UniqueName: \"kubernetes.io/projected/8fc8282b-5810-45a0-885d-47a0915b3a68-kube-api-access-xm9ww\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:04:40.301069 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.300891    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fc8282b-5810-45a0-885d-47a0915b3a68-maas-api-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:04:40.345814 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.345778    2579 generic.go:356] "Generic (PLEG): container finished" podID="8fc8282b-5810-45a0-885d-47a0915b3a68" containerID="e24c769224da6cdbcba4d42f35916b00706b67db7a75739ddf21debf0ac7b8a4" exitCode=0
Sep 19 11:04:40.345946 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.345821    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn" event={"ID":"8fc8282b-5810-45a0-885d-47a0915b3a68","Type":"ContainerDied","Data":"e24c769224da6cdbcba4d42f35916b00706b67db7a75739ddf21debf0ac7b8a4"}
Sep 19 11:04:40.345946 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.345824    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"
Sep 19 11:04:40.345946 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.345865    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn" event={"ID":"8fc8282b-5810-45a0-885d-47a0915b3a68","Type":"ContainerDied","Data":"b48cf7cdfbb8ffc1c063871c9f066e31508c27aade74950fe7691d8d5993b3f9"}
Sep 19 11:04:40.345946 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.345891    2579 scope.go:117] "RemoveContainer" containerID="e24c769224da6cdbcba4d42f35916b00706b67db7a75739ddf21debf0ac7b8a4"
Sep 19 11:04:40.355902 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.355884    2579 scope.go:117] "RemoveContainer" containerID="e24c769224da6cdbcba4d42f35916b00706b67db7a75739ddf21debf0ac7b8a4"
Sep 19 11:04:40.356170 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:04:40.356148    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e24c769224da6cdbcba4d42f35916b00706b67db7a75739ddf21debf0ac7b8a4\": container with ID starting with e24c769224da6cdbcba4d42f35916b00706b67db7a75739ddf21debf0ac7b8a4 not found: ID does not exist" containerID="e24c769224da6cdbcba4d42f35916b00706b67db7a75739ddf21debf0ac7b8a4"
Sep 19 11:04:40.356217 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.356179    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e24c769224da6cdbcba4d42f35916b00706b67db7a75739ddf21debf0ac7b8a4"} err="failed to get container status \"e24c769224da6cdbcba4d42f35916b00706b67db7a75739ddf21debf0ac7b8a4\": rpc error: code = NotFound desc = could not find container \"e24c769224da6cdbcba4d42f35916b00706b67db7a75739ddf21debf0ac7b8a4\": container with ID starting with e24c769224da6cdbcba4d42f35916b00706b67db7a75739ddf21debf0ac7b8a4 not found: ID does not exist"
Sep 19 11:04:40.367142 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.367111    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"]
Sep 19 11:04:40.374381 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.374361    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f210d9-74c646dc66-qrvzn"]
Sep 19 11:04:40.444242 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.444211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:40.539136 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:40.539110    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:41.116313 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:41.116280    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fc8282b-5810-45a0-885d-47a0915b3a68" path="/var/lib/kubelet/pods/8fc8282b-5810-45a0-885d-47a0915b3a68/volumes"
Sep 19 11:04:41.171143 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:41.171114    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:41.231017 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:41.230996    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 11:04:41.236862 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:41.236842    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 11:04:41.781083 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:41.781050    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:43.587507 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:43.587471    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:04:46.263688 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:46.263650    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:46.318406 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:46.318375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:46.582958 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:46.582873    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:47.754660 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:47.754625    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:47.852971 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:47.852933    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:48.364105 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:48.364073    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:48.963803 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:48.963769    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:57.246411 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.246348    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"]
Sep 19 11:04:57.249833 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:04:57.249278    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-serving-cert: secret "maas-api-e2e-shared-a-w2-72e262-serving-cert" not found
Sep 19 11:04:57.249833 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:04:57.249355    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-maas-api-tls podName:0fec2d06-6168-4622-a51e-d2417ee14d5f nodeName:}" failed. No retries permitted until 2026-09-19 11:04:57.74933475 +0000 UTC m=+1818.661028261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-maas-api-tls") pod "maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4" (UID: "0fec2d06-6168-4622-a51e-d2417ee14d5f") : secret "maas-api-e2e-shared-a-w2-72e262-serving-cert" not found
Sep 19 11:04:57.280936 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.280913    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:57.381843 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.381815    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:57.438333 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.438271    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4" podUID="0fec2d06-6168-4622-a51e-d2417ee14d5f" containerName="maas-api" containerID="cri-o://3177957f6ca836b6e393a3983dbb7b13f3d94945a40369acd8874b10699e15c4" gracePeriod=30
Sep 19 11:04:57.457437 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.457418    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:57.514805 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.514750    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:57.678465 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.678441    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 11:04:57.752559 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.752511    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-maas-api-tls\") pod \"0fec2d06-6168-4622-a51e-d2417ee14d5f\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") "
Sep 19 11:04:57.752734 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.752591    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fec2d06-6168-4622-a51e-d2417ee14d5f-openshift-service-ca\") pod \"0fec2d06-6168-4622-a51e-d2417ee14d5f\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") "
Sep 19 11:04:57.752734 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.752631    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-splwd\" (UniqueName: \"kubernetes.io/projected/0fec2d06-6168-4622-a51e-d2417ee14d5f-kube-api-access-splwd\") pod \"0fec2d06-6168-4622-a51e-d2417ee14d5f\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") "
Sep 19 11:04:57.752734 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.752654    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-metrics-certs\") pod \"0fec2d06-6168-4622-a51e-d2417ee14d5f\" (UID: \"0fec2d06-6168-4622-a51e-d2417ee14d5f\") "
Sep 19 11:04:57.752992 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.752968    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fec2d06-6168-4622-a51e-d2417ee14d5f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0fec2d06-6168-4622-a51e-d2417ee14d5f" (UID: "0fec2d06-6168-4622-a51e-d2417ee14d5f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:04:57.754831 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.754802    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fec2d06-6168-4622-a51e-d2417ee14d5f-kube-api-access-splwd" (OuterVolumeSpecName: "kube-api-access-splwd") pod "0fec2d06-6168-4622-a51e-d2417ee14d5f" (UID: "0fec2d06-6168-4622-a51e-d2417ee14d5f"). InnerVolumeSpecName "kube-api-access-splwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:04:57.754936 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.754836    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0fec2d06-6168-4622-a51e-d2417ee14d5f" (UID: "0fec2d06-6168-4622-a51e-d2417ee14d5f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:04:57.754936 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.754874    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0fec2d06-6168-4622-a51e-d2417ee14d5f" (UID: "0fec2d06-6168-4622-a51e-d2417ee14d5f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:04:57.853850 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.853763    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-maas-api-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:04:57.853850 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.853798    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fec2d06-6168-4622-a51e-d2417ee14d5f-openshift-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:04:57.853850 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.853808    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-splwd\" (UniqueName: \"kubernetes.io/projected/0fec2d06-6168-4622-a51e-d2417ee14d5f-kube-api-access-splwd\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:04:57.853850 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.853816    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fec2d06-6168-4622-a51e-d2417ee14d5f-metrics-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:04:57.951208 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:57.951175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:58.006514 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:58.006483    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:58.445032 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:58.444996    2579 generic.go:356] "Generic (PLEG): container finished" podID="0fec2d06-6168-4622-a51e-d2417ee14d5f" containerID="3177957f6ca836b6e393a3983dbb7b13f3d94945a40369acd8874b10699e15c4" exitCode=0
Sep 19 11:04:58.445463 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:58.445040    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"
Sep 19 11:04:58.445463 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:58.445086    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4" event={"ID":"0fec2d06-6168-4622-a51e-d2417ee14d5f","Type":"ContainerDied","Data":"3177957f6ca836b6e393a3983dbb7b13f3d94945a40369acd8874b10699e15c4"}
Sep 19 11:04:58.445463 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:58.445135    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4" event={"ID":"0fec2d06-6168-4622-a51e-d2417ee14d5f","Type":"ContainerDied","Data":"0a84ba88b0bfba8219750f9213048eebf71ca23c96f3cb7cbf596a111fb4837f"}
Sep 19 11:04:58.445463 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:58.445163    2579 scope.go:117] "RemoveContainer" containerID="3177957f6ca836b6e393a3983dbb7b13f3d94945a40369acd8874b10699e15c4"
Sep 19 11:04:58.454835 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:58.454818    2579 scope.go:117] "RemoveContainer" containerID="3177957f6ca836b6e393a3983dbb7b13f3d94945a40369acd8874b10699e15c4"
Sep 19 11:04:58.455097 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:04:58.455076    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3177957f6ca836b6e393a3983dbb7b13f3d94945a40369acd8874b10699e15c4\": container with ID starting with 3177957f6ca836b6e393a3983dbb7b13f3d94945a40369acd8874b10699e15c4 not found: ID does not exist" containerID="3177957f6ca836b6e393a3983dbb7b13f3d94945a40369acd8874b10699e15c4"
Sep 19 11:04:58.455159 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:58.455105    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3177957f6ca836b6e393a3983dbb7b13f3d94945a40369acd8874b10699e15c4"} err="failed to get container status \"3177957f6ca836b6e393a3983dbb7b13f3d94945a40369acd8874b10699e15c4\": rpc error: code = NotFound desc = could not find container \"3177957f6ca836b6e393a3983dbb7b13f3d94945a40369acd8874b10699e15c4\": container with ID starting with 3177957f6ca836b6e393a3983dbb7b13f3d94945a40369acd8874b10699e15c4 not found: ID does not exist"
Sep 19 11:04:58.466743 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:58.466722    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"]
Sep 19 11:04:58.470272 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:58.470247    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-72e262-79bd6f7949-bngm4"]
Sep 19 11:04:58.486156 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:58.486131    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:04:59.115292 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:04:59.115260    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fec2d06-6168-4622-a51e-d2417ee14d5f" path="/var/lib/kubelet/pods/0fec2d06-6168-4622-a51e-d2417ee14d5f/volumes"
Sep 19 11:05:00.294241 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:00.294208    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:05:03.715955 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:03.715922    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:03.775346 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:03.775315    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:03.982745 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:03.982667    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:04.580104 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:04.580077    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:05.361924 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:05.361892    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:05.863098 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:05.863066    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:14.712118 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:14.712081    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"]
Sep 19 11:05:14.712603 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:14.712304    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p" podUID="e7bd7059-22d0-468f-a405-cb5ee2056c37" containerName="maas-api" containerID="cri-o://590b8f91bbf90c581afd4dca84679a52c74a5c3331abb21b83d2cf6a4c72a71d" gracePeriod=30
Sep 19 11:05:14.732879 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:14.732852    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:14.800232 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:14.800205    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:14.853997 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:14.853968    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:14.955956 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:14.955932    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:14.956785 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:14.956770    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 11:05:15.056335 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.056261    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:15.106868 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.106840    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-metrics-certs\") pod \"e7bd7059-22d0-468f-a405-cb5ee2056c37\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") "
Sep 19 11:05:15.106997 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.106905    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6s5gx\" (UniqueName: \"kubernetes.io/projected/e7bd7059-22d0-468f-a405-cb5ee2056c37-kube-api-access-6s5gx\") pod \"e7bd7059-22d0-468f-a405-cb5ee2056c37\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") "
Sep 19 11:05:15.106997 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.106936    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-maas-api-tls\") pod \"e7bd7059-22d0-468f-a405-cb5ee2056c37\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") "
Sep 19 11:05:15.107070 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.107001    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7bd7059-22d0-468f-a405-cb5ee2056c37-openshift-service-ca\") pod \"e7bd7059-22d0-468f-a405-cb5ee2056c37\" (UID: \"e7bd7059-22d0-468f-a405-cb5ee2056c37\") "
Sep 19 11:05:15.107367 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.107339    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7bd7059-22d0-468f-a405-cb5ee2056c37-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e7bd7059-22d0-468f-a405-cb5ee2056c37" (UID: "e7bd7059-22d0-468f-a405-cb5ee2056c37"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:05:15.109122 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.109093    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e7bd7059-22d0-468f-a405-cb5ee2056c37" (UID: "e7bd7059-22d0-468f-a405-cb5ee2056c37"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:05:15.109122 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.109113    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e7bd7059-22d0-468f-a405-cb5ee2056c37" (UID: "e7bd7059-22d0-468f-a405-cb5ee2056c37"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:05:15.109264 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.109102    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7bd7059-22d0-468f-a405-cb5ee2056c37-kube-api-access-6s5gx" (OuterVolumeSpecName: "kube-api-access-6s5gx") pod "e7bd7059-22d0-468f-a405-cb5ee2056c37" (UID: "e7bd7059-22d0-468f-a405-cb5ee2056c37"). InnerVolumeSpecName "kube-api-access-6s5gx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:05:15.208087 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.208063    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7bd7059-22d0-468f-a405-cb5ee2056c37-openshift-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:05:15.208087 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.208089    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-metrics-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:05:15.208222 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.208103    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6s5gx\" (UniqueName: \"kubernetes.io/projected/e7bd7059-22d0-468f-a405-cb5ee2056c37-kube-api-access-6s5gx\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:05:15.208222 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.208115    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7bd7059-22d0-468f-a405-cb5ee2056c37-maas-api-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:05:15.239868 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.239846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:15.408966 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.408904    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:15.542198 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.542163    2579 generic.go:356] "Generic (PLEG): container finished" podID="e7bd7059-22d0-468f-a405-cb5ee2056c37" containerID="590b8f91bbf90c581afd4dca84679a52c74a5c3331abb21b83d2cf6a4c72a71d" exitCode=0
Sep 19 11:05:15.542351 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.542217    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"
Sep 19 11:05:15.542351 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.542252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p" event={"ID":"e7bd7059-22d0-468f-a405-cb5ee2056c37","Type":"ContainerDied","Data":"590b8f91bbf90c581afd4dca84679a52c74a5c3331abb21b83d2cf6a4c72a71d"}
Sep 19 11:05:15.542351 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.542337    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p" event={"ID":"e7bd7059-22d0-468f-a405-cb5ee2056c37","Type":"ContainerDied","Data":"34235a6d25a62f4712ec8b84d875539e6f14d0354f9b85a6fbf45f622b787887"}
Sep 19 11:05:15.542515 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.542364    2579 scope.go:117] "RemoveContainer" containerID="590b8f91bbf90c581afd4dca84679a52c74a5c3331abb21b83d2cf6a4c72a71d"
Sep 19 11:05:15.551219 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.551203    2579 scope.go:117] "RemoveContainer" containerID="590b8f91bbf90c581afd4dca84679a52c74a5c3331abb21b83d2cf6a4c72a71d"
Sep 19 11:05:15.551478 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:05:15.551458    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"590b8f91bbf90c581afd4dca84679a52c74a5c3331abb21b83d2cf6a4c72a71d\": container with ID starting with 590b8f91bbf90c581afd4dca84679a52c74a5c3331abb21b83d2cf6a4c72a71d not found: ID does not exist" containerID="590b8f91bbf90c581afd4dca84679a52c74a5c3331abb21b83d2cf6a4c72a71d"
Sep 19 11:05:15.551543 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.551488    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"590b8f91bbf90c581afd4dca84679a52c74a5c3331abb21b83d2cf6a4c72a71d"} err="failed to get container status \"590b8f91bbf90c581afd4dca84679a52c74a5c3331abb21b83d2cf6a4c72a71d\": rpc error: code = NotFound desc = could not find container \"590b8f91bbf90c581afd4dca84679a52c74a5c3331abb21b83d2cf6a4c72a71d\": container with ID starting with 590b8f91bbf90c581afd4dca84679a52c74a5c3331abb21b83d2cf6a4c72a71d not found: ID does not exist"
Sep 19 11:05:15.559634 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.559612    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"]
Sep 19 11:05:15.564651 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:15.564631    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-99b43e-9b7d4f88-shc2p"]
Sep 19 11:05:17.114557 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:17.114499    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7bd7059-22d0-468f-a405-cb5ee2056c37" path="/var/lib/kubelet/pods/e7bd7059-22d0-468f-a405-cb5ee2056c37/volumes"
Sep 19 11:05:17.494157 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:17.494120    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:05:21.124150 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:21.124118    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:21.229961 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:21.229912    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:21.390789 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:21.390717    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:05:25.696393 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:25.696355    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:05:58.799653 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:05:58.799513    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:06:18.529216 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:06:18.529170    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:06:19.899303 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:06:19.899252    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:06:19.997950 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:06:19.997907    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:06:41.394584 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:06:41.394545    2579 scope.go:117] "RemoveContainer" containerID="642b2e05c7c8c0fec9bfa6fdd81064fe24c9cfcd544d556ce93d5fd5e52e2b59"
Sep 19 11:06:46.194611 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:06:46.194574    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:06:46.825662 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:06:46.825617    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:06:48.381141 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:06:48.381106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:07:12.693271 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:07:12.693233    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:07:20.298706 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:07:20.298670    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:07:28.293077 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:07:28.292997    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:07:46.491241 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:07:46.491205    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:07:54.495772 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:07:54.495736    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:08:49.348365 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:08:49.348327    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:09:20.789415 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:20.789376    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:09:28.384244 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:28.384208    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:09:35.593902 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:35.593861    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:09:40.799432 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:40.799400    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:09:41.264230 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:41.264205    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 11:09:41.280447 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:41.280419    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wzspq_ba18fe0c-ca67-4702-8160-e4490b615f24/ovn-acl-logging/0.log"
Sep 19 11:09:50.224002 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.223964    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"]
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224469    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce267735-7ee2-460e-8d42-90dfc136118a" containerName="main"
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224490    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce267735-7ee2-460e-8d42-90dfc136118a" containerName="main"
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224515    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fc8282b-5810-45a0-885d-47a0915b3a68" containerName="maas-api"
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224543    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fc8282b-5810-45a0-885d-47a0915b3a68" containerName="maas-api"
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224555    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7bd7059-22d0-468f-a405-cb5ee2056c37" containerName="maas-api"
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224564    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7bd7059-22d0-468f-a405-cb5ee2056c37" containerName="maas-api"
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224580    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fec2d06-6168-4622-a51e-d2417ee14d5f" containerName="maas-api"
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224588    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fec2d06-6168-4622-a51e-d2417ee14d5f" containerName="maas-api"
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224611    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b80f0a37-607a-45f4-98ff-f5442875988e" containerName="maas-api"
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224621    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b80f0a37-607a-45f4-98ff-f5442875988e" containerName="maas-api"
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224741    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fec2d06-6168-4622-a51e-d2417ee14d5f" containerName="maas-api"
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224753    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce267735-7ee2-460e-8d42-90dfc136118a" containerName="main"
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224764    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7bd7059-22d0-468f-a405-cb5ee2056c37" containerName="maas-api"
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224775    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fc8282b-5810-45a0-885d-47a0915b3a68" containerName="maas-api"
Sep 19 11:09:50.226417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.224785    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b80f0a37-607a-45f4-98ff-f5442875988e" containerName="maas-api"
Sep 19 11:09:50.230185 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.230160    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.233370 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.233346    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-f6efb1d9-kserve-self-signed-certs\""
Sep 19 11:09:50.238231 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.238206    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"]
Sep 19 11:09:50.261804 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.261782    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:09:50.307201 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.307180    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-dshm\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.307304 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.307225    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-home\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.307304 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.307253    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-model-cache\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.307386 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.307322    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a3e654b-99a3-4331-bfd7-991a670b4d39-tls-certs\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.307386 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.307357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-tmp-dir\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.307386 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.307378    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzqg2\" (UniqueName: \"kubernetes.io/projected/7a3e654b-99a3-4331-bfd7-991a670b4d39-kube-api-access-gzqg2\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.408708 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.408681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-home\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.408845 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.408721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-model-cache\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.408845 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.408743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a3e654b-99a3-4331-bfd7-991a670b4d39-tls-certs\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.408971 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.408864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-tmp-dir\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.408971 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.408912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzqg2\" (UniqueName: \"kubernetes.io/projected/7a3e654b-99a3-4331-bfd7-991a670b4d39-kube-api-access-gzqg2\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.409076 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.408989    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-dshm\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.409194 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.409170    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-home\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.409263 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.409198    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-tmp-dir\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.409263 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.409173    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-model-cache\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.411131 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.411090    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-dshm\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.411264 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.411246    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a3e654b-99a3-4331-bfd7-991a670b4d39-tls-certs\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.416446 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.416428    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzqg2\" (UniqueName: \"kubernetes.io/projected/7a3e654b-99a3-4331-bfd7-991a670b4d39-kube-api-access-gzqg2\") pod \"e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") " pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.544206 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.544131    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:09:50.813995 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.813961    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"]
Sep 19 11:09:50.815575 ip-10-0-140-105 kubenswrapper[2579]: W0919 11:09:50.815514    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a3e654b_99a3_4331_bfd7_991a670b4d39.slice/crio-0a1363124634a4543422b9d173129fa56ee3c3f1262e0d50e42e2b9b775bb591 WatchSource:0}: Error finding container 0a1363124634a4543422b9d173129fa56ee3c3f1262e0d50e42e2b9b775bb591: Status 404 returned error can't find the container with id 0a1363124634a4543422b9d173129fa56ee3c3f1262e0d50e42e2b9b775bb591
Sep 19 11:09:50.926093 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.926058    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s" event={"ID":"7a3e654b-99a3-4331-bfd7-991a670b4d39","Type":"ContainerStarted","Data":"abacb494687cc190fdf7c2e039b6114d0b5b85c2ffaa66770c969e92c817f625"}
Sep 19 11:09:50.926246 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.926104    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s" event={"ID":"7a3e654b-99a3-4331-bfd7-991a670b4d39","Type":"ContainerStarted","Data":"0a1363124634a4543422b9d173129fa56ee3c3f1262e0d50e42e2b9b775bb591"}
Sep 19 11:09:50.940563 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:50.940493    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s" podStartSLOduration=0.940478642 podStartE2EDuration="940.478642ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:09:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:09:50.940066945 +0000 UTC m=+2111.851760461" watchObservedRunningTime="2026-09-19 11:09:50.940478642 +0000 UTC m=+2111.852172161"
Sep 19 11:09:52.694830 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:09:52.694785    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:10:00.545013 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:00.544974    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:10:00.545409 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:00.545031    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:10:00.557597 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:00.557566    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:10:00.994801 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:00.994771    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:10:02.127613 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.127582    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"]
Sep 19 11:10:02.133598 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.131851    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.136788 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.136766    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-f6efb1d9-kserve-self-signed-certs\""
Sep 19 11:10:02.142132 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.142105    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"]
Sep 19 11:10:02.166917 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.166890    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:10:02.215056 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.215020    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-model-cache\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.215237 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.215081    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-tls-certs\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.215237 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.215183    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-home\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.215314 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.215236    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-tmp-dir\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.215314 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.215279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-dshm\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.215314 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.215307    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xccqr\" (UniqueName: \"kubernetes.io/projected/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-kube-api-access-xccqr\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.315963 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.315926    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-tmp-dir\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.315963 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.315969    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-dshm\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.316180 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.316087    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xccqr\" (UniqueName: \"kubernetes.io/projected/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-kube-api-access-xccqr\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.316237 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.316180    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-model-cache\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.316297 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.316234    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-tls-certs\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.316348 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.316303    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-home\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.316414 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.316392    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-tmp-dir\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.316595 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.316565    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-model-cache\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.316595 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.316589    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-home\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.318237 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.318216    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-dshm\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.318648 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.318632    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-tls-certs\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.323786 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.323757    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xccqr\" (UniqueName: \"kubernetes.io/projected/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-kube-api-access-xccqr\") pod \"e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") " pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.447430 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.447394    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:02.718915 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.718884    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"]
Sep 19 11:10:02.720087 ip-10-0-140-105 kubenswrapper[2579]: W0919 11:10:02.720061    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31ba8f1d_5fb4_4500_a79b_48bd34cc7679.slice/crio-c8297db30d252092db4058204e42d0f1f70974863d84e4448433c2ef5c257a44 WatchSource:0}: Error finding container c8297db30d252092db4058204e42d0f1f70974863d84e4448433c2ef5c257a44: Status 404 returned error can't find the container with id c8297db30d252092db4058204e42d0f1f70974863d84e4448433c2ef5c257a44
Sep 19 11:10:02.995590 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.995479    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f" event={"ID":"31ba8f1d-5fb4-4500-a79b-48bd34cc7679","Type":"ContainerStarted","Data":"b08dc84b3ccd7a873d241d0f7fdf6bbf583ef71e171fef5d3af8121f020a0fbb"}
Sep 19 11:10:02.995590 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:02.995535    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f" event={"ID":"31ba8f1d-5fb4-4500-a79b-48bd34cc7679","Type":"ContainerStarted","Data":"c8297db30d252092db4058204e42d0f1f70974863d84e4448433c2ef5c257a44"}
Sep 19 11:10:03.011597 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:03.011542    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f" podStartSLOduration=1.011513614 podStartE2EDuration="1.011513614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:10:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:10:03.009551156 +0000 UTC m=+2123.921244669" watchObservedRunningTime="2026-09-19 11:10:03.011513614 +0000 UTC m=+2123.923207159"
Sep 19 11:10:04.997201 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:04.997160    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:10:12.448221 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:12.448177    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:12.448221 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:12.448235    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:12.460761 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:12.460739    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:13.060299 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:13.060263    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:22.957775 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:22.957742    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"]
Sep 19 11:10:22.976047 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:22.976024    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:10:23.969076 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:23.969040    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"]
Sep 19 11:10:23.985720 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:23.985667    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:10:26.097447 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:26.097415    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:10:35.388948 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:35.388914    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:10:37.977040 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:37.976981    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s" podUID="7a3e654b-99a3-4331-bfd7-991a670b4d39" containerName="main" containerID="cri-o://abacb494687cc190fdf7c2e039b6114d0b5b85c2ffaa66770c969e92c817f625" gracePeriod=45
Sep 19 11:10:38.183229 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.183198    2579 generic.go:356] "Generic (PLEG): container finished" podID="7a3e654b-99a3-4331-bfd7-991a670b4d39" containerID="abacb494687cc190fdf7c2e039b6114d0b5b85c2ffaa66770c969e92c817f625" exitCode=0
Sep 19 11:10:38.183389 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.183284    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s" event={"ID":"7a3e654b-99a3-4331-bfd7-991a670b4d39","Type":"ContainerDied","Data":"abacb494687cc190fdf7c2e039b6114d0b5b85c2ffaa66770c969e92c817f625"}
Sep 19 11:10:38.224484 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.224462    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:10:38.329899 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.329819    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-tmp-dir\") pod \"7a3e654b-99a3-4331-bfd7-991a670b4d39\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") "
Sep 19 11:10:38.329899 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.329865    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzqg2\" (UniqueName: \"kubernetes.io/projected/7a3e654b-99a3-4331-bfd7-991a670b4d39-kube-api-access-gzqg2\") pod \"7a3e654b-99a3-4331-bfd7-991a670b4d39\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") "
Sep 19 11:10:38.329899 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.329882    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a3e654b-99a3-4331-bfd7-991a670b4d39-tls-certs\") pod \"7a3e654b-99a3-4331-bfd7-991a670b4d39\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") "
Sep 19 11:10:38.330125 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.329971    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-dshm\") pod \"7a3e654b-99a3-4331-bfd7-991a670b4d39\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") "
Sep 19 11:10:38.330125 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.330020    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-home\") pod \"7a3e654b-99a3-4331-bfd7-991a670b4d39\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") "
Sep 19 11:10:38.330125 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.330066    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-model-cache\") pod \"7a3e654b-99a3-4331-bfd7-991a670b4d39\" (UID: \"7a3e654b-99a3-4331-bfd7-991a670b4d39\") "
Sep 19 11:10:38.330288 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.330120    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a3e654b-99a3-4331-bfd7-991a670b4d39" (UID: "7a3e654b-99a3-4331-bfd7-991a670b4d39"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:38.330347 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.330295    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-home" (OuterVolumeSpecName: "home") pod "7a3e654b-99a3-4331-bfd7-991a670b4d39" (UID: "7a3e654b-99a3-4331-bfd7-991a670b4d39"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:38.330347 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.330314    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a3e654b-99a3-4331-bfd7-991a670b4d39" (UID: "7a3e654b-99a3-4331-bfd7-991a670b4d39"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:38.330347 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.330310    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:38.332131 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.332096    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-dshm" (OuterVolumeSpecName: "dshm") pod "7a3e654b-99a3-4331-bfd7-991a670b4d39" (UID: "7a3e654b-99a3-4331-bfd7-991a670b4d39"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:38.332252 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.332167    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a3e654b-99a3-4331-bfd7-991a670b4d39-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a3e654b-99a3-4331-bfd7-991a670b4d39" (UID: "7a3e654b-99a3-4331-bfd7-991a670b4d39"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:10:38.332252 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.332196    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a3e654b-99a3-4331-bfd7-991a670b4d39-kube-api-access-gzqg2" (OuterVolumeSpecName: "kube-api-access-gzqg2") pod "7a3e654b-99a3-4331-bfd7-991a670b4d39" (UID: "7a3e654b-99a3-4331-bfd7-991a670b4d39"). InnerVolumeSpecName "kube-api-access-gzqg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:10:38.430888 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.430857    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:38.430888 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.430881    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:38.430888 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.430890    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a3e654b-99a3-4331-bfd7-991a670b4d39-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:38.431102 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.430900    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzqg2\" (UniqueName: \"kubernetes.io/projected/7a3e654b-99a3-4331-bfd7-991a670b4d39-kube-api-access-gzqg2\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:38.431102 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.430909    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a3e654b-99a3-4331-bfd7-991a670b4d39-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:38.989086 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:38.989015    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f" podUID="31ba8f1d-5fb4-4500-a79b-48bd34cc7679" containerName="main" containerID="cri-o://b08dc84b3ccd7a873d241d0f7fdf6bbf583ef71e171fef5d3af8121f020a0fbb" gracePeriod=45
Sep 19 11:10:39.190130 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.190102    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"
Sep 19 11:10:39.190288 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.190131    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s" event={"ID":"7a3e654b-99a3-4331-bfd7-991a670b4d39","Type":"ContainerDied","Data":"0a1363124634a4543422b9d173129fa56ee3c3f1262e0d50e42e2b9b775bb591"}
Sep 19 11:10:39.190288 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.190186    2579 scope.go:117] "RemoveContainer" containerID="abacb494687cc190fdf7c2e039b6114d0b5b85c2ffaa66770c969e92c817f625"
Sep 19 11:10:39.192119 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.192013    2579 generic.go:356] "Generic (PLEG): container finished" podID="31ba8f1d-5fb4-4500-a79b-48bd34cc7679" containerID="b08dc84b3ccd7a873d241d0f7fdf6bbf583ef71e171fef5d3af8121f020a0fbb" exitCode=0
Sep 19 11:10:39.192119 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.192049    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f" event={"ID":"31ba8f1d-5fb4-4500-a79b-48bd34cc7679","Type":"ContainerDied","Data":"b08dc84b3ccd7a873d241d0f7fdf6bbf583ef71e171fef5d3af8121f020a0fbb"}
Sep 19 11:10:39.205918 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.205894    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"]
Sep 19 11:10:39.211001 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.210979    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-f6efb1d9-kserve-5849f5b97f-7559s"]
Sep 19 11:10:39.227757 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.227739    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:39.338642 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.338572    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-tmp-dir\") pod \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") "
Sep 19 11:10:39.338642 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.338615    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xccqr\" (UniqueName: \"kubernetes.io/projected/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-kube-api-access-xccqr\") pod \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") "
Sep 19 11:10:39.338824 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.338646    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-home\") pod \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") "
Sep 19 11:10:39.338824 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.338676    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-model-cache\") pod \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") "
Sep 19 11:10:39.338824 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.338724    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-tls-certs\") pod \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") "
Sep 19 11:10:39.338824 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.338794    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-dshm\") pod \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\" (UID: \"31ba8f1d-5fb4-4500-a79b-48bd34cc7679\") "
Sep 19 11:10:39.339054 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.338855    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "31ba8f1d-5fb4-4500-a79b-48bd34cc7679" (UID: "31ba8f1d-5fb4-4500-a79b-48bd34cc7679"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:39.339054 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.338902    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-home" (OuterVolumeSpecName: "home") pod "31ba8f1d-5fb4-4500-a79b-48bd34cc7679" (UID: "31ba8f1d-5fb4-4500-a79b-48bd34cc7679"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:39.339054 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.338918    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-model-cache" (OuterVolumeSpecName: "model-cache") pod "31ba8f1d-5fb4-4500-a79b-48bd34cc7679" (UID: "31ba8f1d-5fb4-4500-a79b-48bd34cc7679"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:39.339225 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.339058    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-home\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:39.339225 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.339071    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-model-cache\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:39.339225 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.339081    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-tmp-dir\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:39.340769 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.340747    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-dshm" (OuterVolumeSpecName: "dshm") pod "31ba8f1d-5fb4-4500-a79b-48bd34cc7679" (UID: "31ba8f1d-5fb4-4500-a79b-48bd34cc7679"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 11:10:39.340839 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.340750    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "31ba8f1d-5fb4-4500-a79b-48bd34cc7679" (UID: "31ba8f1d-5fb4-4500-a79b-48bd34cc7679"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:10:39.340883 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.340851    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-kube-api-access-xccqr" (OuterVolumeSpecName: "kube-api-access-xccqr") pod "31ba8f1d-5fb4-4500-a79b-48bd34cc7679" (UID: "31ba8f1d-5fb4-4500-a79b-48bd34cc7679"). InnerVolumeSpecName "kube-api-access-xccqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:10:39.439718 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.439685    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-tls-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:39.439718 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.439714    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-dshm\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:39.439879 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:39.439724    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xccqr\" (UniqueName: \"kubernetes.io/projected/31ba8f1d-5fb4-4500-a79b-48bd34cc7679-kube-api-access-xccqr\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:40.198801 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.198761    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"
Sep 19 11:10:40.198801 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.198797    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f" event={"ID":"31ba8f1d-5fb4-4500-a79b-48bd34cc7679","Type":"ContainerDied","Data":"c8297db30d252092db4058204e42d0f1f70974863d84e4448433c2ef5c257a44"}
Sep 19 11:10:40.199303 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.198838    2579 scope.go:117] "RemoveContainer" containerID="b08dc84b3ccd7a873d241d0f7fdf6bbf583ef71e171fef5d3af8121f020a0fbb"
Sep 19 11:10:40.219443 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.219417    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"]
Sep 19 11:10:40.221434 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.221409    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-f6efb1d9-kserve-74c7c89f99-plh8f"]
Sep 19 11:10:40.488616 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.488538    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:10:40.769405 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.769328    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"]
Sep 19 11:10:40.769700 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.769687    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31ba8f1d-5fb4-4500-a79b-48bd34cc7679" containerName="main"
Sep 19 11:10:40.769741 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.769702    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="31ba8f1d-5fb4-4500-a79b-48bd34cc7679" containerName="main"
Sep 19 11:10:40.769741 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.769711    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a3e654b-99a3-4331-bfd7-991a670b4d39" containerName="main"
Sep 19 11:10:40.769741 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.769717    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a3e654b-99a3-4331-bfd7-991a670b4d39" containerName="main"
Sep 19 11:10:40.769844 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.769796    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="31ba8f1d-5fb4-4500-a79b-48bd34cc7679" containerName="main"
Sep 19 11:10:40.769844 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.769808    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a3e654b-99a3-4331-bfd7-991a670b4d39" containerName="main"
Sep 19 11:10:40.774089 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.774063    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:40.776459 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.776441    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 19 11:10:40.776713 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.776695    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 11:10:40.777605 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.777588    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 11:10:40.777679 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.777641    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 11:10:40.777740 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.777710    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8b2hp\""
Sep 19 11:10:40.784663 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.784544    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"]
Sep 19 11:10:40.852117 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.852088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slsxw\" (UniqueName: \"kubernetes.io/projected/fe5a23d7-30ef-40b8-98d8-0d056c619ede-kube-api-access-slsxw\") pod \"maas-api-f47b4794c-6wx7c\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") " pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:40.852258 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.852138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe5a23d7-30ef-40b8-98d8-0d056c619ede-maas-api-tls\") pod \"maas-api-f47b4794c-6wx7c\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") " pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:40.852258 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.852220    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe5a23d7-30ef-40b8-98d8-0d056c619ede-openshift-service-ca\") pod \"maas-api-f47b4794c-6wx7c\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") " pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:40.852349 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.852289    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe5a23d7-30ef-40b8-98d8-0d056c619ede-metrics-certs\") pod \"maas-api-f47b4794c-6wx7c\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") " pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:40.952869 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.952833    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe5a23d7-30ef-40b8-98d8-0d056c619ede-metrics-certs\") pod \"maas-api-f47b4794c-6wx7c\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") " pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:40.952869 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.952873    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slsxw\" (UniqueName: \"kubernetes.io/projected/fe5a23d7-30ef-40b8-98d8-0d056c619ede-kube-api-access-slsxw\") pod \"maas-api-f47b4794c-6wx7c\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") " pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:40.953113 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.952897    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe5a23d7-30ef-40b8-98d8-0d056c619ede-maas-api-tls\") pod \"maas-api-f47b4794c-6wx7c\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") " pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:40.953113 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.952928    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe5a23d7-30ef-40b8-98d8-0d056c619ede-openshift-service-ca\") pod \"maas-api-f47b4794c-6wx7c\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") " pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:40.953566 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.953544    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe5a23d7-30ef-40b8-98d8-0d056c619ede-openshift-service-ca\") pod \"maas-api-f47b4794c-6wx7c\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") " pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:40.955242 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.955218    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe5a23d7-30ef-40b8-98d8-0d056c619ede-maas-api-tls\") pod \"maas-api-f47b4794c-6wx7c\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") " pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:40.955389 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.955367    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe5a23d7-30ef-40b8-98d8-0d056c619ede-metrics-certs\") pod \"maas-api-f47b4794c-6wx7c\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") " pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:40.960533 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:40.960499    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slsxw\" (UniqueName: \"kubernetes.io/projected/fe5a23d7-30ef-40b8-98d8-0d056c619ede-kube-api-access-slsxw\") pod \"maas-api-f47b4794c-6wx7c\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") " pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:41.088347 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:41.088264    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8b2hp\""
Sep 19 11:10:41.096383 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:41.096354    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:41.118806 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:41.118770    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31ba8f1d-5fb4-4500-a79b-48bd34cc7679" path="/var/lib/kubelet/pods/31ba8f1d-5fb4-4500-a79b-48bd34cc7679/volumes"
Sep 19 11:10:41.119357 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:41.119332    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a3e654b-99a3-4331-bfd7-991a670b4d39" path="/var/lib/kubelet/pods/7a3e654b-99a3-4331-bfd7-991a670b4d39/volumes"
Sep 19 11:10:41.370701 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:41.370674    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"]
Sep 19 11:10:41.372386 ip-10-0-140-105 kubenswrapper[2579]: W0919 11:10:41.372352    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe5a23d7_30ef_40b8_98d8_0d056c619ede.slice/crio-b069704faedc3ab7ff50d94adb7d1880ed45408307b2834546eecdeb5a65a99b WatchSource:0}: Error finding container b069704faedc3ab7ff50d94adb7d1880ed45408307b2834546eecdeb5a65a99b: Status 404 returned error can't find the container with id b069704faedc3ab7ff50d94adb7d1880ed45408307b2834546eecdeb5a65a99b
Sep 19 11:10:41.373950 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:41.373930    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 11:10:42.215937 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:42.215902    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c" event={"ID":"fe5a23d7-30ef-40b8-98d8-0d056c619ede","Type":"ContainerStarted","Data":"add040ab0c71b5cfb096d19e315e48781807f9e761e2e1f2bf73f2ee4eb1f9a7"}
Sep 19 11:10:42.216117 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:42.215953    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c" event={"ID":"fe5a23d7-30ef-40b8-98d8-0d056c619ede","Type":"ContainerStarted","Data":"b069704faedc3ab7ff50d94adb7d1880ed45408307b2834546eecdeb5a65a99b"}
Sep 19 11:10:42.216117 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:42.216000    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:42.232859 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:42.232815    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c" podStartSLOduration=1.795078247 podStartE2EDuration="2.23280329s" totalImagesPullingTime="437.725043ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:10:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:10:42.230868488 +0000 UTC m=+2163.142562007" watchObservedRunningTime="2026-09-19 11:10:42.23280329 +0000 UTC m=+2163.144496809"
Sep 19 11:10:47.996114 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:47.996074    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"]
Sep 19 11:10:47.996611 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:47.996322    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c" podUID="fe5a23d7-30ef-40b8-98d8-0d056c619ede" containerName="maas-api" containerID="cri-o://add040ab0c71b5cfb096d19e315e48781807f9e761e2e1f2bf73f2ee4eb1f9a7" gracePeriod=30
Sep 19 11:10:48.002444 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.002423    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:48.026872 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.026837    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"]
Sep 19 11:10:48.030178 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.030159    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:48.037968 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.037946    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"]
Sep 19 11:10:48.218157 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.218131    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edbfbc53-6126-4104-9d52-04409b9021bc-maas-api-tls\") pod \"maas-api-546445c5bc-wh45g\" (UID: \"edbfbc53-6126-4104-9d52-04409b9021bc\") " pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:48.218289 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.218271    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfmnd\" (UniqueName: \"kubernetes.io/projected/edbfbc53-6126-4104-9d52-04409b9021bc-kube-api-access-vfmnd\") pod \"maas-api-546445c5bc-wh45g\" (UID: \"edbfbc53-6126-4104-9d52-04409b9021bc\") " pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:48.218347 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.218315    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edbfbc53-6126-4104-9d52-04409b9021bc-openshift-service-ca\") pod \"maas-api-546445c5bc-wh45g\" (UID: \"edbfbc53-6126-4104-9d52-04409b9021bc\") " pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:48.218500 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.218476    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edbfbc53-6126-4104-9d52-04409b9021bc-metrics-certs\") pod \"maas-api-546445c5bc-wh45g\" (UID: \"edbfbc53-6126-4104-9d52-04409b9021bc\") " pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:48.236532 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.236499    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:48.248012 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.247951    2579 generic.go:356] "Generic (PLEG): container finished" podID="fe5a23d7-30ef-40b8-98d8-0d056c619ede" containerID="add040ab0c71b5cfb096d19e315e48781807f9e761e2e1f2bf73f2ee4eb1f9a7" exitCode=0
Sep 19 11:10:48.248012 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.247984    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"
Sep 19 11:10:48.248146 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.248031    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c" event={"ID":"fe5a23d7-30ef-40b8-98d8-0d056c619ede","Type":"ContainerDied","Data":"add040ab0c71b5cfb096d19e315e48781807f9e761e2e1f2bf73f2ee4eb1f9a7"}
Sep 19 11:10:48.248146 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.248079    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c" event={"ID":"fe5a23d7-30ef-40b8-98d8-0d056c619ede","Type":"ContainerDied","Data":"b069704faedc3ab7ff50d94adb7d1880ed45408307b2834546eecdeb5a65a99b"}
Sep 19 11:10:48.248146 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.248101    2579 scope.go:117] "RemoveContainer" containerID="add040ab0c71b5cfb096d19e315e48781807f9e761e2e1f2bf73f2ee4eb1f9a7"
Sep 19 11:10:48.257199 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.257179    2579 scope.go:117] "RemoveContainer" containerID="add040ab0c71b5cfb096d19e315e48781807f9e761e2e1f2bf73f2ee4eb1f9a7"
Sep 19 11:10:48.257628 ip-10-0-140-105 kubenswrapper[2579]: E0919 11:10:48.257558    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"add040ab0c71b5cfb096d19e315e48781807f9e761e2e1f2bf73f2ee4eb1f9a7\": container with ID starting with add040ab0c71b5cfb096d19e315e48781807f9e761e2e1f2bf73f2ee4eb1f9a7 not found: ID does not exist" containerID="add040ab0c71b5cfb096d19e315e48781807f9e761e2e1f2bf73f2ee4eb1f9a7"
Sep 19 11:10:48.257628 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.257593    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"add040ab0c71b5cfb096d19e315e48781807f9e761e2e1f2bf73f2ee4eb1f9a7"} err="failed to get container status \"add040ab0c71b5cfb096d19e315e48781807f9e761e2e1f2bf73f2ee4eb1f9a7\": rpc error: code = NotFound desc = could not find container \"add040ab0c71b5cfb096d19e315e48781807f9e761e2e1f2bf73f2ee4eb1f9a7\": container with ID starting with add040ab0c71b5cfb096d19e315e48781807f9e761e2e1f2bf73f2ee4eb1f9a7 not found: ID does not exist"
Sep 19 11:10:48.294235 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.294199    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:10:48.319842 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.319812    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edbfbc53-6126-4104-9d52-04409b9021bc-maas-api-tls\") pod \"maas-api-546445c5bc-wh45g\" (UID: \"edbfbc53-6126-4104-9d52-04409b9021bc\") " pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:48.319978 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.319846    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfmnd\" (UniqueName: \"kubernetes.io/projected/edbfbc53-6126-4104-9d52-04409b9021bc-kube-api-access-vfmnd\") pod \"maas-api-546445c5bc-wh45g\" (UID: \"edbfbc53-6126-4104-9d52-04409b9021bc\") " pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:48.319978 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.319872    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edbfbc53-6126-4104-9d52-04409b9021bc-openshift-service-ca\") pod \"maas-api-546445c5bc-wh45g\" (UID: \"edbfbc53-6126-4104-9d52-04409b9021bc\") " pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:48.319978 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.319900    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edbfbc53-6126-4104-9d52-04409b9021bc-metrics-certs\") pod \"maas-api-546445c5bc-wh45g\" (UID: \"edbfbc53-6126-4104-9d52-04409b9021bc\") " pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:48.320440 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.320419    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edbfbc53-6126-4104-9d52-04409b9021bc-openshift-service-ca\") pod \"maas-api-546445c5bc-wh45g\" (UID: \"edbfbc53-6126-4104-9d52-04409b9021bc\") " pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:48.322262 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.322239    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/edbfbc53-6126-4104-9d52-04409b9021bc-metrics-certs\") pod \"maas-api-546445c5bc-wh45g\" (UID: \"edbfbc53-6126-4104-9d52-04409b9021bc\") " pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:48.322371 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.322353    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edbfbc53-6126-4104-9d52-04409b9021bc-maas-api-tls\") pod \"maas-api-546445c5bc-wh45g\" (UID: \"edbfbc53-6126-4104-9d52-04409b9021bc\") " pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:48.330296 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.330274    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfmnd\" (UniqueName: \"kubernetes.io/projected/edbfbc53-6126-4104-9d52-04409b9021bc-kube-api-access-vfmnd\") pod \"maas-api-546445c5bc-wh45g\" (UID: \"edbfbc53-6126-4104-9d52-04409b9021bc\") " pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:48.377288 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.377264    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:48.420742 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.420710    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe5a23d7-30ef-40b8-98d8-0d056c619ede-openshift-service-ca\") pod \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") "
Sep 19 11:10:48.420929 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.420770    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slsxw\" (UniqueName: \"kubernetes.io/projected/fe5a23d7-30ef-40b8-98d8-0d056c619ede-kube-api-access-slsxw\") pod \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") "
Sep 19 11:10:48.420929 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.420826    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe5a23d7-30ef-40b8-98d8-0d056c619ede-maas-api-tls\") pod \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") "
Sep 19 11:10:48.420929 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.420845    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe5a23d7-30ef-40b8-98d8-0d056c619ede-metrics-certs\") pod \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\" (UID: \"fe5a23d7-30ef-40b8-98d8-0d056c619ede\") "
Sep 19 11:10:48.421107 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.421080    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe5a23d7-30ef-40b8-98d8-0d056c619ede-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fe5a23d7-30ef-40b8-98d8-0d056c619ede" (UID: "fe5a23d7-30ef-40b8-98d8-0d056c619ede"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 11:10:48.421190 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.421171    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe5a23d7-30ef-40b8-98d8-0d056c619ede-openshift-service-ca\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:48.423349 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.423324    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe5a23d7-30ef-40b8-98d8-0d056c619ede-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fe5a23d7-30ef-40b8-98d8-0d056c619ede" (UID: "fe5a23d7-30ef-40b8-98d8-0d056c619ede"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:10:48.423575 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.423471    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe5a23d7-30ef-40b8-98d8-0d056c619ede-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fe5a23d7-30ef-40b8-98d8-0d056c619ede" (UID: "fe5a23d7-30ef-40b8-98d8-0d056c619ede"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 11:10:48.423660 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.423637    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe5a23d7-30ef-40b8-98d8-0d056c619ede-kube-api-access-slsxw" (OuterVolumeSpecName: "kube-api-access-slsxw") pod "fe5a23d7-30ef-40b8-98d8-0d056c619ede" (UID: "fe5a23d7-30ef-40b8-98d8-0d056c619ede"). InnerVolumeSpecName "kube-api-access-slsxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 11:10:48.522473 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.522386    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slsxw\" (UniqueName: \"kubernetes.io/projected/fe5a23d7-30ef-40b8-98d8-0d056c619ede-kube-api-access-slsxw\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:48.522473 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.522425    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe5a23d7-30ef-40b8-98d8-0d056c619ede-maas-api-tls\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:48.522473 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.522435    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe5a23d7-30ef-40b8-98d8-0d056c619ede-metrics-certs\") on node \"ip-10-0-140-105.ec2.internal\" DevicePath \"\""
Sep 19 11:10:48.569943 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.569919    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"]
Sep 19 11:10:48.577446 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.577422    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-f47b4794c-6wx7c"]
Sep 19 11:10:48.648121 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:48.648098    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"]
Sep 19 11:10:48.649382 ip-10-0-140-105 kubenswrapper[2579]: W0919 11:10:48.649357    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedbfbc53_6126_4104_9d52_04409b9021bc.slice/crio-85adf552fed834c3e333b74f4440fa5cea856dd5a86766052d66c9b4c6735d82 WatchSource:0}: Error finding container 85adf552fed834c3e333b74f4440fa5cea856dd5a86766052d66c9b4c6735d82: Status 404 returned error can't find the container with id 85adf552fed834c3e333b74f4440fa5cea856dd5a86766052d66c9b4c6735d82
Sep 19 11:10:49.116226 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:49.116192    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe5a23d7-30ef-40b8-98d8-0d056c619ede" path="/var/lib/kubelet/pods/fe5a23d7-30ef-40b8-98d8-0d056c619ede/volumes"
Sep 19 11:10:49.255474 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:49.255392    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g" event={"ID":"edbfbc53-6126-4104-9d52-04409b9021bc","Type":"ContainerStarted","Data":"3b23d1b01f2b883e5ac01f3d303b9e37b9c8d992a4904164874e7e27309e3c00"}
Sep 19 11:10:49.255474 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:49.255456    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g" event={"ID":"edbfbc53-6126-4104-9d52-04409b9021bc","Type":"ContainerStarted","Data":"85adf552fed834c3e333b74f4440fa5cea856dd5a86766052d66c9b4c6735d82"}
Sep 19 11:10:49.255683 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:49.255478    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:10:49.270794 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:49.270670    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g" podStartSLOduration=0.985040439 podStartE2EDuration="1.270653986s" totalImagesPullingTime="285.613547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 11:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:10:49.270158574 +0000 UTC m=+2170.181852096" watchObservedRunningTime="2026-09-19 11:10:49.270653986 +0000 UTC m=+2170.182347504"
Sep 19 11:10:54.990026 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:54.989983    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:10:55.265882 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:10:55.265810    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-546445c5bc-wh45g"
Sep 19 11:11:00.794015 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:00.793966    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:11:07.696031 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:07.695991    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-6xxmd"]
Sep 19 11:11:11.071419 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:11.071359    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:11:11.120284 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:11.120253    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:11:16.906403 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:16.906370    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b87889df-lwjlp_51b7401b-5d3d-4fdf-a9ea-8d735e333095/copy-manifests/0.log"
Sep 19 11:11:16.918356 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:16.918327    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b87889df-lwjlp_51b7401b-5d3d-4fdf-a9ea-8d735e333095/manager/0.log"
Sep 19 11:11:17.596098 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:17.596071    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-546445c5bc-wh45g_edbfbc53-6126-4104-9d52-04409b9021bc/maas-api/0.log"
Sep 19 11:11:17.700213 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:17.700183    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29830245-n42n5_71efe150-94ce-413c-98e1-29704b2587c7/cleanup/0.log"
Sep 19 11:11:18.997719 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:18.997694    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-lkvnw_add13f26-9889-4f49-b89d-df992594a021/kuadrant-console-plugin/0.log"
Sep 19 11:11:19.224099 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:19.224069    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-6xxmd_13ce7aab-198d-412a-9c20-c5f5ca6fac1e/limitador/0.log"
Sep 19 11:11:20.016893 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:20.016863    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8b786b9b5-wppgk_4af6e02a-6920-412e-9baa-86ff466cdf24/kube-auth-proxy/0.log"
Sep 19 11:11:21.030417 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:21.030389    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh_5f6db6b8-47c6-44f9-8c29-f3a203ba192f/main/0.log"
Sep 19 11:11:21.237982 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:21.237952    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6_60c115c6-2bfa-47eb-81ba-6620bb7239cc/main/0.log"
Sep 19 11:11:21.447399 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:21.447373    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt_5fa6796b-89df-460b-9275-0683e443326a/main/0.log"
Sep 19 11:11:29.364014 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:29.363987    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b87889df-lwjlp_51b7401b-5d3d-4fdf-a9ea-8d735e333095/copy-manifests/0.log"
Sep 19 11:11:29.375033 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:29.375011    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b87889df-lwjlp_51b7401b-5d3d-4fdf-a9ea-8d735e333095/manager/0.log"
Sep 19 11:11:30.066762 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:30.066727    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-546445c5bc-wh45g_edbfbc53-6126-4104-9d52-04409b9021bc/maas-api/0.log"
Sep 19 11:11:30.173047 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:30.173017    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29830245-n42n5_71efe150-94ce-413c-98e1-29704b2587c7/cleanup/0.log"
Sep 19 11:11:31.466422 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:31.466396    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-lkvnw_add13f26-9889-4f49-b89d-df992594a021/kuadrant-console-plugin/0.log"
Sep 19 11:11:31.715020 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:31.714990    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-6xxmd_13ce7aab-198d-412a-9c20-c5f5ca6fac1e/limitador/0.log"
Sep 19 11:11:32.530592 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:32.530562    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8b786b9b5-wppgk_4af6e02a-6920-412e-9baa-86ff466cdf24/kube-auth-proxy/0.log"
Sep 19 11:11:33.622774 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:33.622743    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-fxcwh_5f6db6b8-47c6-44f9-8c29-f3a203ba192f/main/0.log"
Sep 19 11:11:33.837279 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:33.837240    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqn8r6_60c115c6-2bfa-47eb-81ba-6620bb7239cc/main/0.log"
Sep 19 11:11:34.066432 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:34.066400    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-zlzzt_5fa6796b-89df-460b-9275-0683e443326a/main/0.log"
Sep 19 11:11:52.924458 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:52.924426    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:11:52.984238 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:52.984212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log" limit=1024
Sep 19 11:11:55.694064 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:55.694032    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vr6wb_630f71c6-b3e9-4935-8c99-921f92d7494c/global-pull-secret-syncer/0.log"
Sep 19 11:11:55.736103 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:55.736068    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5xl48_347ba7f0-42f9-43c2-b54f-55466a818064/konnectivity-agent/0.log"
Sep 19 11:11:55.818201 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:11:55.818173    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-105.ec2.internal_b8679eb2ddf3de7904fb5919a04512a3/haproxy/0.log"
Sep 19 11:12:01.252316 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:01.252290    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-lkvnw_add13f26-9889-4f49-b89d-df992594a021/kuadrant-console-plugin/0.log"
Sep 19 11:12:01.331092 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:01.331062    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-6xxmd_13ce7aab-198d-412a-9c20-c5f5ca6fac1e/limitador/0.log"
Sep 19 11:12:02.942639 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:02.942596    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7674648dc8-9hb4n_47c4dd98-e96a-426c-a40b-821612719f6a/metrics-server/0.log"
Sep 19 11:12:02.972078 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:02.972051    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-sm5wr_98fa1395-052d-4cb6-a5cc-e889c967b1a2/monitoring-plugin/0.log"
Sep 19 11:12:03.094901 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:03.094824    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gzxgf_aac09470-a013-4090-9983-ad5fb9f90b2a/node-exporter/0.log"
Sep 19 11:12:03.113571 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:03.113549    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gzxgf_aac09470-a013-4090-9983-ad5fb9f90b2a/kube-rbac-proxy/0.log"
Sep 19 11:12:03.137361 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:03.137340    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gzxgf_aac09470-a013-4090-9983-ad5fb9f90b2a/init-textfile/0.log"
Sep 19 11:12:03.230410 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:03.230369    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-lb88l_a0f9227b-ae29-4410-968b-f7e2ce416ffa/kube-rbac-proxy-main/0.log"
Sep 19 11:12:03.248731 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:03.248699    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-lb88l_a0f9227b-ae29-4410-968b-f7e2ce416ffa/kube-rbac-proxy-self/0.log"
Sep 19 11:12:03.266517 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:03.266487    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-lb88l_a0f9227b-ae29-4410-968b-f7e2ce416ffa/openshift-state-metrics/0.log"
Sep 19 11:12:03.529842 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:03.529810    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69b884cc84-bmrcv_f8a20344-2487-4643-856f-67df241098bb/telemeter-client/0.log"
Sep 19 11:12:03.548000 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:03.547976    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69b884cc84-bmrcv_f8a20344-2487-4643-856f-67df241098bb/reload/0.log"
Sep 19 11:12:03.580649 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:03.580618    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69b884cc84-bmrcv_f8a20344-2487-4643-856f-67df241098bb/kube-rbac-proxy/0.log"
Sep 19 11:12:05.741708 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.741675    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"]
Sep 19 11:12:05.742084 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.741994    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe5a23d7-30ef-40b8-98d8-0d056c619ede" containerName="maas-api"
Sep 19 11:12:05.742084 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.742005    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe5a23d7-30ef-40b8-98d8-0d056c619ede" containerName="maas-api"
Sep 19 11:12:05.742084 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.742080    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe5a23d7-30ef-40b8-98d8-0d056c619ede" containerName="maas-api"
Sep 19 11:12:05.746706 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.746687    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.749236 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.749211    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mc4ck\"/\"kube-root-ca.crt\""
Sep 19 11:12:05.749236 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.749225    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mc4ck\"/\"openshift-service-ca.crt\""
Sep 19 11:12:05.750722 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.750700    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mc4ck\"/\"default-dockercfg-rpzfz\""
Sep 19 11:12:05.753466 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.753434    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"]
Sep 19 11:12:05.872731 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.872702    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f01dc1b-e7c5-4527-af36-e55105a39f96-lib-modules\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.872872 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.872752    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q2sc\" (UniqueName: \"kubernetes.io/projected/6f01dc1b-e7c5-4527-af36-e55105a39f96-kube-api-access-2q2sc\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.872872 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.872826    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f01dc1b-e7c5-4527-af36-e55105a39f96-podres\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.872872 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.872851    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f01dc1b-e7c5-4527-af36-e55105a39f96-sys\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.872986 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.872890    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f01dc1b-e7c5-4527-af36-e55105a39f96-proc\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.973598 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.973569    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f01dc1b-e7c5-4527-af36-e55105a39f96-lib-modules\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.973748 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.973619    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q2sc\" (UniqueName: \"kubernetes.io/projected/6f01dc1b-e7c5-4527-af36-e55105a39f96-kube-api-access-2q2sc\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.973748 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.973653    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f01dc1b-e7c5-4527-af36-e55105a39f96-podres\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.973748 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.973672    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f01dc1b-e7c5-4527-af36-e55105a39f96-sys\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.973748 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.973700    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f01dc1b-e7c5-4527-af36-e55105a39f96-proc\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.973960 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.973767    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f01dc1b-e7c5-4527-af36-e55105a39f96-proc\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.973960 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.973770    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f01dc1b-e7c5-4527-af36-e55105a39f96-lib-modules\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.973960 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.973782    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f01dc1b-e7c5-4527-af36-e55105a39f96-podres\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.973960 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.973799    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f01dc1b-e7c5-4527-af36-e55105a39f96-sys\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:05.981261 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:05.981236    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q2sc\" (UniqueName: \"kubernetes.io/projected/6f01dc1b-e7c5-4527-af36-e55105a39f96-kube-api-access-2q2sc\") pod \"perf-node-gather-daemonset-9wqff\" (UID: \"6f01dc1b-e7c5-4527-af36-e55105a39f96\") " pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:06.057278 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:06.057192    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:06.217318 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:06.217290    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-xjb6l_90b5fc61-0581-47ff-a61b-31cd617c1352/download-server/0.log"
Sep 19 11:12:06.325087 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:06.325062    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"]
Sep 19 11:12:06.327754 ip-10-0-140-105 kubenswrapper[2579]: W0919 11:12:06.327724    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6f01dc1b_e7c5_4527_af36_e55105a39f96.slice/crio-890c5e809a96958da77276cbdac16d324c7c65bda0b1f65cd71fb1ca816d6dcf WatchSource:0}: Error finding container 890c5e809a96958da77276cbdac16d324c7c65bda0b1f65cd71fb1ca816d6dcf: Status 404 returned error can't find the container with id 890c5e809a96958da77276cbdac16d324c7c65bda0b1f65cd71fb1ca816d6dcf
Sep 19 11:12:06.646019 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:06.645931    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff" event={"ID":"6f01dc1b-e7c5-4527-af36-e55105a39f96","Type":"ContainerStarted","Data":"b1033187161f6a0b213b20379d67c25e56f707447524d3cc60152b9b37952a9a"}
Sep 19 11:12:06.646019 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:06.645988    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:06.646019 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:06.646006    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff" event={"ID":"6f01dc1b-e7c5-4527-af36-e55105a39f96","Type":"ContainerStarted","Data":"890c5e809a96958da77276cbdac16d324c7c65bda0b1f65cd71fb1ca816d6dcf"}
Sep 19 11:12:06.661392 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:06.661349    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff" podStartSLOduration=1.661333296 podStartE2EDuration="1.661333296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 11:12:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 11:12:06.659690984 +0000 UTC m=+2247.571384503" watchObservedRunningTime="2026-09-19 11:12:06.661333296 +0000 UTC m=+2247.573026823"
Sep 19 11:12:07.900726 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:07.900696    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jqhh6_d6349d68-d16e-4a28-b8c1-40630155ed16/dns/0.log"
Sep 19 11:12:07.918284 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:07.918264    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jqhh6_d6349d68-d16e-4a28-b8c1-40630155ed16/kube-rbac-proxy/0.log"
Sep 19 11:12:08.019085 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:08.019059    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wr84k_5492514c-c9bf-4516-9314-d1adda92b9ff/dns-node-resolver/0.log"
Sep 19 11:12:08.521158 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:08.521120    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-56b557bc79-sqmkg_d54dedf1-5f91-45c6-b218-5d2d342a962e/registry/0.log"
Sep 19 11:12:08.557824 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:08.557801    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qrxfh_2c21f7fa-a5d2-4c47-9e12-7ee4a905e870/node-ca/0.log"
Sep 19 11:12:09.820077 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:09.820048    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8b786b9b5-wppgk_4af6e02a-6920-412e-9baa-86ff466cdf24/kube-auth-proxy/0.log"
Sep 19 11:12:10.800237 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:10.800210    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q7nq7_0b45f9d0-487d-41c2-97e0-98865a07f066/serve-healthcheck-canary/0.log"
Sep 19 11:12:11.239051 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:11.239018    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-66fmf_e8659512-e949-4ab4-a636-7d0e84f95048/kube-rbac-proxy/0.log"
Sep 19 11:12:11.256178 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:11.256160    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-66fmf_e8659512-e949-4ab4-a636-7d0e84f95048/exporter/0.log"
Sep 19 11:12:11.273746 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:11.273725    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-66fmf_e8659512-e949-4ab4-a636-7d0e84f95048/extractor/0.log"
Sep 19 11:12:12.660789 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:12.660761    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mc4ck/perf-node-gather-daemonset-9wqff"
Sep 19 11:12:14.122108 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:14.122074    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b87889df-lwjlp_51b7401b-5d3d-4fdf-a9ea-8d735e333095/manager/0.log"
Sep 19 11:12:14.137003 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:14.136977    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9b87889df-lwjlp_51b7401b-5d3d-4fdf-a9ea-8d735e333095/copy-manifests/0.log"
Sep 19 11:12:15.668276 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:15.668200    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6c7d4469d9-2j6qn_d3fd3981-41ea-4832-a60f-b1ad893fd18f/manager/0.log"
Sep 19 11:12:16.517500 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:16.517473    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8b786b9b5-wppgk_4af6e02a-6920-412e-9baa-86ff466cdf24/kube-auth-proxy/0.log"
Sep 19 11:12:18.586596 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:18.586563    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8b786b9b5-wppgk_4af6e02a-6920-412e-9baa-86ff466cdf24/kube-auth-proxy/0.log"
Sep 19 11:12:20.629156 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:20.629128    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-jq5z7_98e60d0a-95a5-4f4a-9b82-64148a71fe7a/migrator/0.log"
Sep 19 11:12:20.645684 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:20.645659    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-jq5z7_98e60d0a-95a5-4f4a-9b82-64148a71fe7a/graceful-termination/0.log"
Sep 19 11:12:21.878019 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:21.877989    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8vvd2_44ddc6bf-b4b4-48f5-90a8-a6a6e98e3dae/kube-multus/0.log"
Sep 19 11:12:22.145920 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:22.145844    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p4shs_fd50bc98-a748-4587-aacf-56daf296fe88/kube-multus-additional-cni-plugins/0.log"
Sep 19 11:12:22.163678 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:22.163652    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p4shs_fd50bc98-a748-4587-aacf-56daf296fe88/egress-router-binary-copy/0.log"
Sep 19 11:12:22.180823 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:22.180805    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p4shs_fd50bc98-a748-4587-aacf-56daf296fe88/cni-plugins/0.log"
Sep 19 11:12:22.198110 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:22.198093    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p4shs_fd50bc98-a748-4587-aacf-56daf296fe88/bond-cni-plugin/0.log"
Sep 19 11:12:22.214363 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:22.214342    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p4shs_fd50bc98-a748-4587-aacf-56daf296fe88/routeoverride-cni/0.log"
Sep 19 11:12:22.233867 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:22.233846    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p4shs_fd50bc98-a748-4587-aacf-56daf296fe88/whereabouts-cni-bincopy/0.log"
Sep 19 11:12:22.250476 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:22.250455    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p4shs_fd50bc98-a748-4587-aacf-56daf296fe88/whereabouts-cni/0.log"
Sep 19 11:12:22.459531 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:22.459490    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lnqqz_f85be02b-a097-4f90-844c-21d54e7dc53d/network-metrics-daemon/0.log"
Sep 19 11:12:22.474576 ip-10-0-140-105 kubenswrapper[2579]: I0919 11:12:22.474549    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lnqqz_f85be02b-a097-4f90-844c-21d54e7dc53d/kube-rbac-proxy/0.log"